How to train INSANELY fast JEV Decision Models from scratch (architecture deep-dive)
Neural Breakdown with AVB
36,016 views
An implementation of SystemOne Decision Models inspired by Jev. Check Prime Intellect here: https://www.primeintellect.ai/?utm_so...
Follow me on Twitter: https://x.com/neural_avb
Support on Patreon: Patreon: NeuralBreakdownwithAVB
Members get access to everything behind-the-scenes that goes into producing my videos - including code. Plus, it supports the channel in a big way and helps to pay my bills.
Training repo: https://github.com/avbiswas/bev-train
HF Dataset: https://huggingface.co/datasets/avbis...
Model: https://huggingface.co/avbiswas/bev-d...
Livesteam coding: https://youtube.com/live/AzxoU7kxjig
Timestamp:
0:00 Intro: System One decision models
1:38: What is Jev?
3:43 What we're building
8:08 Choice, score & noul questions
12:25 Bi-encoders vs joint encoders
14:26 Designing the network on Qwen
17:55 The decision head
22:49 Making it choice-order invariant
32:39 Live coding the model in PyTorch
1:31:46 Building the dataset
1:41:35 Training & results
20 Comments
How to train INSANELY fast JEV Decision Models from scratch (architecture deep-dive)
Neural Breakdown with AVB
36,016 views
An implementation of SystemOne Decision Models inspired by Jev. Check Prime Intellect here: https://www.primeintellect.ai/?utm_so...
Follow me on Twitter: https://x.com/neural_avb
Support on Patreon: Patreon: NeuralBreakdownwithAVB
Members get access to everything behind-the-scenes that goes into producing my videos - including code. Plus, it supports the channel in a big way and helps to pay my bills.
Training repo: https://github.com/avbiswas/bev-train
HF Dataset: https://huggingface.co/datasets/avbis...
Model: https://huggingface.co/avbiswas/bev-d...
Livesteam coding: https://youtube.com/live/AzxoU7kxjig
Timestamp:
0:00 Intro: System One decision models
1:38: What is Jev?
3:43 What we're building
8:08 Choice, score & noul questions
12:25 Bi-encoders vs joint encoders
14:26 Designing the network on Qwen
17:55 The decision head
22:49 Making it choice-order invariant
32:39 Live coding the model in PyTorch
1:31:46 Building the dataset
1:41:35 Training & results