[Phase 5: CAEC + CFCP] Pipeline Monitor

⚠️ ATTENTION: Fold 0, Fold 1, Fold 2, Fold 3, Fold 4 IS NOT RUNNING!

Process appears stopped/crashed. Resume command(s):

python src/run_phase5_cfcp.py --step all --folds 0 --search-seeds 42 --device cuda --resume
python src/run_phase5_cfcp.py --step all --folds 1 --search-seeds 42 --device cuda --resume
python src/run_phase5_cfcp.py --step all --folds 2 --search-seeds 42 --device cuda --resume
python src/run_phase5_cfcp.py --step all --folds 3 --search-seeds 42 --device cuda --resume
python src/run_phase5_cfcp.py --step all --folds 4 --search-seeds 42 --device cuda --resume

Pipeline Health

0/5 Active (5 STOPPED)

Folds Status

Est. Pipeline Finish

Fri 10-02 10:13

ETA: 1d 22h (Fold 1)

GPU Compute

99%

Fan: 79% | Bus: 24%

Dedicated VRAM

7.5 / 10.0 GB (75%)

RTX 3080 GDDR6X

Shared GPU RAM

0.45 GB

Clean (Zero Spillover)

GPU Thermal & Power

58°C

233.3 W Draw

CPU Usage & Temp

19.1%

Temp: --

Host RAM

21.2 / 31.9 GB (66%)

System Memory

Snapshot Time: 2026-09-30 11:28:17 [Phase 5: CAEC + CFCP | Dir: phase5_caec_cfcp] ============================================================================================================================================================================== Fold | Phase | Status | Gen | Cand/Gen | Current Epoch | Req/Uniq | CacheHit | Guided% | Best F1 | WorstRec | Sec/Ep | ETA | Est Finish ============================================================================================================================================================================== 0 | Search | STOPPED | 6/30 | 19/20 | 3f195cd7 (2/5) | 100/91 | 9.0%(9) | 41.0% | 0.6257 | 0.1362 | 61.9s | 1d 13h | Fri 10-02 01:27 1 | Search | STOPPED | 6/30 | 7/20 | 739a0312 (4/5) | 100/95 | 5.0%(5) | 27.0% | 0.6494 | 0.1288 | 74.5s | 1d 22h | Fri 10-02 10:13 2 | Search | STOPPED | 7/30 | 4/20 | 263d49c9 (1/5) | 120/112 | 6.7%(8) | 40.0% | 0.6433 | 0.1477 | 53.4s | 1d 8h | Thu 10-01 19:54 3 | Search | STOPPED | 5/30 | 7/20 | cbe13928 (5/5) | 80/79 | 1.2%(1) | 32.5% | 0.6556 | 0.1343 | 50.7s | 1d 9h | Thu 10-01 20:33 4 | Search | STOPPED | 5/30 | 12/20 | f793793c (5/5) | 80/76 | 5.0%(4) | 38.8% | 0.6410 | 0.1548 | 52.8s | 1d 10h | Thu 10-01 21:34 ============================================================================================================================================================================== !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [⚠️ ATTENTION: STOPPED / CRASHED WORKERS] * Fold 0 is NOT RUNNING (Status: STOPPED)! - Stopped at: Phase Search, Gen 6/30 - Resume command: python src/run_phase5_cfcp.py --step all --folds 0 --search-seeds 42 --device cuda --resume * Fold 1 is NOT RUNNING (Status: STOPPED)! - Stopped at: Phase Search, Gen 6/30 - Resume command: python src/run_phase5_cfcp.py --step all --folds 1 --search-seeds 42 --device cuda --resume * Fold 2 is NOT RUNNING (Status: STOPPED)! - Stopped at: Phase Search, Gen 7/30 - Resume command: python src/run_phase5_cfcp.py --step all --folds 2 --search-seeds 42 --device cuda --resume * Fold 3 is NOT RUNNING (Status: STOPPED)! - Stopped at: Phase Search, Gen 5/30 - Resume command: python src/run_phase5_cfcp.py --step all --folds 3 --search-seeds 42 --device cuda --resume * Fold 4 is NOT RUNNING (Status: STOPPED)! - Stopped at: Phase Search, Gen 5/30 - Resume command: python src/run_phase5_cfcp.py --step all --folds 4 --search-seeds 42 --device cuda --resume !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [ACTIVE WORKER DETAILS] No active workers currently executing. --------------------------------------------------------------------------------