● LIVE [Phase 5: CAEC + CFCP] Pipeline Monitor
Snapshot: 2026-10-01 09:28:23 | Est. Pipeline Finish: Sat 10-03 07:10
⚠️ ATTENTION: Fold 4 IS NOT RUNNING!
Process appears stopped/crashed. Resume command(s):
python src/run_phase5_cfcp.py --step all --folds 4 --search-seeds 42 --device cuda --resume
Pipeline Health
4/5 Active (1 STOPPED)
Folds Status
Est. Pipeline Finish
Sat 10-03 07:10
ETA: 1d 21h (Fold 4)
GPU Compute
99%
Fan: 79% | Bus: 14%
Dedicated VRAM
8.3 / 10.0 GB (83%)
RTX 3080 GDDR6X
Shared GPU RAM
0.00 GB
Clean (Zero Spillover)
GPU Thermal & Power
55°C
204.9 W Draw
CPU Usage & Temp
29.6%
Temp: --
Host RAM
18.0 / 31.9 GB (56%)
System Memory
Snapshot Time: 2026-10-01 09:28:24  [Phase 5: CAEC + CFCP | Dir: phase5_caec_cfcp]
==============================================================================================================================================================================
Fold  | Phase      | Status    | Gen     | Cand/Gen   | Current Epoch      | Req/Uniq   | CacheHit   | Guided%  | Best F1  | WorstRec | Sec/Ep  | ETA      | Est Finish       
==============================================================================================================================================================================
0     | Search     | Running   | 17/30   | 5/20       | a05d1164 (4/5)     | 320/273    | 14.7%(47)  | 47.2%    | 0.6277   | 0.1438   | 47.2s   | 15.9h    | Fri 10-02 01:24  
1     | Search     | Running   | 15/30   | 4/20       | 6f7aa1a8 (5/5)     | 280/260    | 7.1%(20)   | 46.1%    | 0.6494   | 0.1367   | 48.6s   | 19.7h    | Fri 10-02 05:12  
2     | Search     | Running   | 17/30   | 7/20       | 961c8425 (2/5)     | 320/263    | 17.8%(57)  | 47.5%    | 0.6433   | 0.1508   | 39.8s   | 12.9h    | Thu 10-01 22:21  
3     | Search     | Running   | 12/30   | 11/20      | 5479a9b3 (3/5)     | 220/202    | 8.2%(18)   | 41.4%    | 0.6556   | 0.1367   | 47.5s   | 22.5h    | Fri 10-02 07:55  
4     | Search     | STOPPED   | 8/30    | 10/20      | 6bb217a2 (1/5)     | 140/133    | 5.0%(7)    | 40.7%    | 0.6446   | 0.1548   | 79.5s   | 1d 21h   | Sat 10-03 07:10  
==============================================================================================================================================================================

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[⚠️ ATTENTION: STOPPED / CRASHED WORKERS]
  * Fold 4 is NOT RUNNING (Status: STOPPED)!
    - Stopped at: Phase Search, Gen 8/30
    - Resume command:
      python src/run_phase5_cfcp.py --step all --folds 4 --search-seeds 42 --device cuda --resume
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[ACTIVE WORKER DETAILS]
  * Fold 0 [RUNNING]:
    - Generation: 17/30 | Active Candidate ID: a05d1164554c0668df546e9c08e7d9945a622f3a6909db6459d9d90df91cb642
    - Training Progress: Epoch 4/5 (measured: 47.2s/epoch)
    - Last Epoch Val: Macro-F1 = 0.5420, Loss = 0.0000, Worst-Rec = 0.0000
  * Fold 1 [RUNNING]:
    - Generation: 15/30 | Active Candidate ID: 6f7aa1a8f457d8f5a5c3de1009b5884f09cd7379379cd467243d55ca2867e5c2
    - Training Progress: Epoch 5/5 (measured: 48.6s/epoch)
    - Last Epoch Val: Macro-F1 = 0.6347, Loss = 0.0000, Worst-Rec = 0.0636
  * Fold 2 [RUNNING]:
    - Generation: 17/30 | Active Candidate ID: 961c84257c088cbb53f71ae97f4cb82bbf65d55e74b67bfd10ed5b965970d86a
    - Training Progress: Epoch 2/5 (measured: 39.8s/epoch)
    - Last Epoch Val: Macro-F1 = 0.5694, Loss = 0.0000, Worst-Rec = 0.1249
  * Fold 3 [RUNNING]:
    - Generation: 12/30 | Active Candidate ID: 5479a9b3507fcb1c4f8b94d67f47b34856738bc0cc3777b69171ff090776cb55
    - Training Progress: Epoch 3/5 (measured: 47.5s/epoch)
    - Last Epoch Val: Macro-F1 = 0.5957, Loss = 0.0000, Worst-Rec = 0.0259
--------------------------------------------------------------------------------