switch to honcho to run processor and cache manager
Categories
(Socorro :: Processor, task, P2)
Tracking
(Not tracked)
People
(Reporter: willkg, Assigned: willkg)
References
(Blocks 3 open bugs)
Details
At startup, the processor kicks off an additional process to clean up the on-disk sym cache. Instead of running it on its own, it'd be better to switch to a process manager like Honcho to run it.
This reduces another complexity with the the processor architecture.
Assignee | ||
Comment 1•4 years ago
|
||
Making this block the GCP migration. I'll do this work before then.
Assignee | ||
Comment 2•3 years ago
|
||
Grabbing this to do soon.
Assignee | ||
Comment 3•2 years ago
|
||
I thought this would be pretty straight-forward.
- write a
bin/run_processor_honcho.sh
that kicks off honcho using environment variables to configure things - write a
processor/Procfile
that runs/app/bin/run_processor.sh
-- we already had this, so nothing changes here - change
bin/entrypoint.sh
to run honcho
I do that and suddenly none of the configman-style environment variables make it to the processor and it dies. I did some debugging:
bin/entrypoint.sh
sees configman-style environment variables in the environment.bin/entrypoint.sh
runs/app/bin/run_processor_honcho.sh
which sees configman-style environment variables in the environment/app/bin/run_processor_honcho.sh
runs honcho which runs/app/bin/run_processor.sh
and the configman-style environment variables have been dropped from the environment
The honcho code has this code which restricts names to [A-Za-z_][A-Za-z_0-9]*
, but I'm pretty sure that only runs on .env
files and we're not using that.
Then I traced how Honcho kicks off processes and what's going on is that it runs subprocess.Popen
which on Linux runs popen_spawn
which I think runs something like /bin/sh
with the command. /bin/sh
on Linux drops invalid variables including configman-style variables.
So... it looks like I have to fix configman stuff first.
Assignee | ||
Comment 4•2 years ago
|
||
I overhauled the configuration in Socorro so we're no longer using configman. That was in bug #1529342. As part of that, I changed the Socorro processor to run with Honcho.
All of that went out today in bug #1824987. Marking as FIXED.
Description
•