-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(UI): New progress bar #3403
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: r3drun3 <simone.ragonesi@sighup.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: John Mastron <14130495+mtronrd@users.noreply.github.com> Co-authored-by: John Mastron <jmastron@jpl.nasa.gov> Co-authored-by: Sergio Garcia <sergargar1@gmail.com> Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com> Co-authored-by: sergargar <sergargar@users.noreply.github.com> Co-authored-by: Pepe Fagoaga <pepe@verica.io> Co-authored-by: github-actions <noreply@github.com> Co-authored-by: simone ragonesi <102741679+R3DRUN3@users.noreply.github.com> Co-authored-by: Johnny Lu <johnny2lu@gmail.com> Co-authored-by: Vajrala Venkateswarlu <59252985+venkyvajrala@users.noreply.github.com> Co-authored-by: Ignacio Dominguez <ignacio.dominguez@zego.com>
removed function param - it is not needed
fixed blank space + removed the if statement in __init__ which I should have previously removed
Co-authored-by: Nacho Rivera <nachor1992@gmail.com>
…#3046) Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
…oud#3252) Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com> Co-authored-by: Sergio Garcia <sergargar1@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: r3drun3 <simone.ragonesi@sighup.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: John Mastron <14130495+mtronrd@users.noreply.github.com> Co-authored-by: John Mastron <jmastron@jpl.nasa.gov> Co-authored-by: Sergio Garcia <sergargar1@gmail.com> Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com> Co-authored-by: sergargar <sergargar@users.noreply.github.com> Co-authored-by: Pepe Fagoaga <pepe@verica.io> Co-authored-by: github-actions <noreply@github.com> Co-authored-by: simone ragonesi <102741679+R3DRUN3@users.noreply.github.com> Co-authored-by: Johnny Lu <johnny2lu@gmail.com> Co-authored-by: Vajrala Venkateswarlu <59252985+venkyvajrala@users.noreply.github.com> Co-authored-by: Ignacio Dominguez <ignacio.dominguez@zego.com>
Co-authored-by: Nacho Rivera <nachor1992@gmail.com>
…#3046) Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
…oud#3252) Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com> Co-authored-by: Sergio Garcia <sergargar1@gmail.com>
sergargar
changed the title
Fennerr new progress bar
feat(UI): New progress bar and ExecutionManager
Feb 15, 2024
sergargar
changed the title
feat(UI): New progress bar and ExecutionManager
feat(UI): New progress bar
Feb 15, 2024
sergargar
added
backport-v3
Pending to port to Prowler v3 branch
and removed
backport-v3
Pending to port to Prowler v3 branch
labels
Mar 14, 2024
Hi @Fennerr just to give you an update, we are working on a new CLI for Prowler and a new Scan Manager inspired in the work you did in this PR. Sadly this PR wouldn't be merged since it got stuck in a previous version. If you want to contribute in the new code please let me know, I'm trying to reutilise your code so feel free to ping me. Thanks for all your time and contributions to Prowler ❤️ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Improved UI for prowler 4.0
Currently only supports the AWS provider.
Description
Switched from using alive_progress bar to using Rich to display the UI
I wanted to implement the new ui without having to update the service class's or checks code.
I was able to do it for the service classes, by modifying the threading_call to initialize the progress bar with X number of tasks (length of the iterator passed to the function), and increment the progress bar as each thread completes it's task.
There is also a progress_decorator to be used where threading_call is not appropriate - like for global services (see list_distributions in the CloudFront service)
Doing checks was not as straightforward, as the logic for the way checks are handled is coded into the execute function, so I couldnt do anything at the base class level. For the most part, checks are really quick, and dont need a progress bar. One place where it could be appropriate, where it does take a long time to execute, was the secrets check for lambda functions (other secrets check are also likely to take a decent amount of time and could benefit from a progress bar)
In these cases, inside the execute function, a call to self.start_task needs to be made with the display message and the number of tasks (self.start_task("Processing functions...", len(functions)) for checking secrets in lambda code). Then after completing an item in the list of items to check, a call to self.increment_task_progress() needs to be made. At the end of the check, just after returning the findings, I made a call to self.update_title_with_findings(findings) so that the UI shows X/Y failed the test. I like how it looks, but it requires some changes to be made to the checks which is what I was trying to avoid - but couldn't find a way around this.
I have also implemented an execution_manager. I thought I had previously pushed a branch with this change, but it doesnt seem to be the case. The execution_manager was made to help be more deliberate about which clients are intialized, then run all the checks that use that client, and then release the client for it to be garage collected.
It determines what clients are going to be imported by all the checks that are going to run (by using /check_to_client_mapper.py. It then starts running through the checks, and will handle initializing any clients needed by the check. It will then prioritize the next check based off of what clients are already loaded, and release the clients once they are no longer needed.
I have also implemented the --only-logs functionality - in which case the live display is not intialized.
This pull request is a WIP, but wanted to provide it so longer so it's easier to see the diff between main and what I have done.
I still need to complete changing the functions inside the service init's to make use of threading_call or progress_decorator, and any other feedback provided.