-
Notifications
You must be signed in to change notification settings - Fork 942
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parallel cannot identify the default shell #5339
Comments
FYI, if I add the flag |
Hi @hajapy! Thanks for reporting this. I tried the steps you posted, and I couldn't reproduce the issue. In my case, lining reports a success. What is your runtime environment like? We have a similar report in #5070 (comment) about running this on a arm64 machine (which we don't support (yet?)) |
I am running on an M1 macbook pro (macOS 14.3.1) with Docker Desktop 4.27.2 (137060) which runs the amd64 image under emulation. |
I see. That looks like the same issue that's reported in the comment I linked. The
I think it makes sense to set it to |
Given that you already cloned the Add the following line after line 396 in ENV SHELL=/bin/bash Then run: make docker
make lint-codebase This should hopefully resolve this particular issue. If so, we can create a PR to implement that. Thanks! |
I gave it a try with
🤷♂️ FWIW, I think I may have a clue why this is happening: https://git.savannah.gnu.org/cgit/parallel.git/tree/src/parallel#n2246 $Global::shell = $ENV{'PARALLEL_SHELL'} || parent_shell($$)
|| $ENV{'SHELL'} || "/bin/sh"; So even defining SHELL env var won't override its priority logic if we're using a different shell which it can determine. However, it seems under emulation parent_shell may not work? cat /proc/$pid/cmdline
/run/rosetta/rosetta/bin/bash/bin/bash also readlink /proc/$$/exe
/run/rosetta/rosetta I looks to me like it doesn't understand the parent shell using its various tricks for docker under rosetta emulation, thus falls back to $SHELL or /bin/sh. |
I see, thanks for going far in triaging this. So, if I read that logic correctly, |
I believe both would work, with PARALLEL_SHELL being a forced override and SHELL being a fallback in case it can't automatically determine the shell correctly. https://www.gnu.org/software/parallel/parallel_design.html#which-shell-to-use
|
Maybe we have better luck with the make IMAGE=slim docker
make lint-codebase |
Also, I'd like to reproduce this in our CI environment somehow, so we can have feedback about the proposed fix. I wasn't able to reproduce this in a Linux-based environment. |
Another observation about |
Yes, this worked.
AFAIK, this is bash-specific. I don't know what (if any) other shells support it. But it is how super-linter is making the function available to parallel: https://github.com/super-linter/super-linter/blob/main/lib/functions/buildFileList.sh#L511
You could configure a macos arm runner to run a test of a built amd64 image under emulation? https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/ |
The GNU parallel manual states, in line with what you observed in its source code:
The shell that started GNU Parallel is
case you listed above. |
Ah! We crossed the streams :)
Yeah, I think so.
Yeah, saw that. We dynamically generate part of the workflow matrix, so I'm thinking about the best way to do this ;) |
Also, there are a few interesting |
I figured out you can add
|
Thanks for these details! To me, this looks like something that Parallel should address, although the lowest hanging fruit might be to explicitly set the |
If you're worried about unintended side effects you could just add the SHELL=/bin/bash when building the parallel command. I'll have a look into created a bug report in the parallel project. Once arm64 images become available this becomes moot, as there shouldn't be as much need to run super-linter under emulation. There is also the workaround of just feeding the env variable in docker run invocation. |
Maybe we can try with |
Is there an existing issue for this?
Current Behavior
Running super-linter images locally fails with multiple error message
/bin/sh: BuildFileArrays: not found
.x-ref: #5335 (comment)
Expected Behavior
The parallel command should not fail in this way and the
/bin/sh: BuildFileArrays: not found
. should not occur.Super-Linter version
Relevant log output
Full log.txt
Steps To Reproduce
6c1f40c
Anything else?
I believe this is occurring because the SHELL environment variable is not set and gnu parallel is choosing to use /bin/sh as a default even though bash is the desired shell. I suspect this can be resolved by exporting env var SHELL=/bin/bash (or setting this in the dockerfile).
Example from testing in the slim-latest image:
The text was updated successfully, but these errors were encountered: