change hook weight to -1 (job has to run after config maps placed)#776
change hook weight to -1 (job has to run after config maps placed)#776
Conversation
| annotations: | ||
| "helm.sh/hook": pre-install,pre-upgrade | ||
| "helm.sh/hook-weight": "-2" | ||
| "helm.sh/hook-weight": "-1" |
There was a problem hiding this comment.
Init and migration jobs now run concurrently
This change correctly ensures the config maps (weight -2) are created before the database_init_job runs. However, before this PR the ordering was:
-2: config maps anddatabase_init_job(concurrent, which was itself the bug being fixed)-1:database_migration_job
After this PR, both database_init_job and database_migration_job share weight -1, meaning Helm will submit and run them concurrently. If db.runDbInitScript and db.runDbMigrationScript are ever both enabled in the same release (e.g., a first-install that also needs migrations), the migration job could start before the init job finishes — which could cause the migration to fail against an uninitialized database.
If these two flags are guaranteed to be mutually exclusive, this is fine. If there is any scenario where both could be true simultaneously, consider bumping database_migration_job to weight 0 to preserve a sequential init → migrate order.
Prompt To Fix With AI
This is a comment left during a code review.
Path: charts/model-engine/templates/database_init_job.yaml
Line: 10
Comment:
**Init and migration jobs now run concurrently**
This change correctly ensures the config maps (weight `-2`) are created before the `database_init_job` runs. However, before this PR the ordering was:
- `-2`: config maps **and** `database_init_job` (concurrent, which was itself the bug being fixed)
- `-1`: `database_migration_job`
After this PR, both `database_init_job` and `database_migration_job` share weight `-1`, meaning Helm will submit and run them **concurrently**. If `db.runDbInitScript` and `db.runDbMigrationScript` are ever both enabled in the same release (e.g., a first-install that also needs migrations), the migration job could start before the init job finishes — which could cause the migration to fail against an uninitialized database.
If these two flags are guaranteed to be mutually exclusive, this is fine. If there is any scenario where both could be `true` simultaneously, consider bumping `database_migration_job` to weight `0` to preserve a sequential init → migrate order.
How can I resolve this? If you propose a fix, please make it concise.
Change hook weight for the job to -1 (the config maps with hook weight -2 need to be created in order for the job to succeed)
Greptile Summary
This PR fixes a Helm hook ordering bug by bumping
database_init_job'shelm.sh/hook-weightfrom-2to-1, ensuring the job starts only after the config maps it depends on (all at weight-2) have been successfully created.-1guarantees it runs after all-2config maps are in place, resolving the root cause described in the PR.database_migration_jobalready sits at weight-1. With this change, both jobs share the same weight and Helm will run them concurrently. Previously the init job ran strictly before the migration job (init at-2, migration at-1). If bothdb.runDbInitScriptanddb.runDbMigrationScriptcan ever betruein the same release, the migration could race against an uninitialized database.Confidence Score: 4/5
db.runDbInitScriptanddb.runDbMigrationScriptare never both enabled simultaneously.Important Files Changed
-2to-1so the job runs after config maps (weight-2) are created. This collapses the previously sequential init→migration ordering sincedatabase_migration_jobis also at-1, but the two jobs are gated by separate flags so concurrent execution is likely not a real-world concern.Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD subgraph weight_minus2 ["Hook Weight: -2 (runs first)"] CM1[service_config_map] CM2[service_template_config_map] CM3[inference_framework_config] CM4[aws_config_map] end subgraph weight_minus1 ["Hook Weight: -1 (after config maps)"] J1[database_init_job ✅ changed from -2] J2[database_migration_job] end subgraph weight_0 ["Hook Weight: 0"] J3[spellbook_init_job] end subgraph weight_1 ["Hook Weight: 1 (runs last)"] J4[restart_keda_operator] J5[populate_fine_tuning_repository_job] end weight_minus2 --> weight_minus1 weight_minus1 --> weight_0 weight_0 --> weight_1 style J1 fill:#90EE90,stroke:#333Prompt To Fix All With AI
Last reviewed commit: 73cda88