Projects
Projects / ProjectZero

ProjectZero

A machine-learning framework written from scratch in C++, with no ML libraries: a feed-forward neural network, two evolutionary trainers, three test environments, a modular agent–environment system and a Qt desktop GUI.

March 2016 – January 2017 UNR graduate research group C++11 · standard library only wjhdfnmf/ProjectZero
~5,700lines of C++ I wrote
10 monthsfrom first network to full framework
4 of 5cart-balance runs succeed
R² ≥ 0.96on 7 of 8 target functions

Overview

Between March 2016 and January 2017 I built UNR Project Zero, a machine-learning framework in C++ written from scratch. I built it at 21 as a member of a University of Nevada, Reno graduate research group led by Prof. Logan Yliniemi. I learned the fundamentals through self-study and group discussion, then wrote the code myself. I'd been programming C++ on Arduino since I was 15.

About 5,700 lines of the C++ are mine. The cart-balance physics was written by a collaborator, Sierra Gonzales, and the GUI's plotting widget is QCustomPlot; everything else is my own code.

Re-run in November 2026, it still works:

4 of 5training runs brought an inverted pendulum from 5° to within 0.01° of upright in about 1 s and held it for the full 20 s test
7 of 8target functions (x²+1, a step, eˣ, sin x, √x+1, −x, a constant) learned to R² 0.96–1.00
94.6%of 1,000 games: the explore/exploit agent found the best of 10 slot machines

The project shows the fundamentals done by hand — a forward pass and weight mutation, tournament selection, input/output scaling and an environment interface, experiment logging and plotting — and steady iteration from v2.1 to v3.0 of the network and trainer within 2016.

Timeline

The core was built in ten months, March 2016 to January 2017.

Timeline from March 2016 to January 2017: neural network v2.1 (Mar 1), trainer v2.1 function fitting (Jul 8), grid world (Sep 13), neural network v3.0 (Oct 1), trainer v3.0 cart balance (Oct 27), agent-environment system 1.1 (Dec 20), system main program (Jan 1)
Start dates from the header comment of each source file in the repository.

The multi-armed bandit and the Qt GUI are from the same period, but their files carry no exact date. The code went to GitHub in December 2022; in March 2025 I updated the plots and added the cart's push force.

What I built

Seven components, all in plain C++11 using only the standard library.

ComponentWhat it doesLinesState in 2026
Neural network v3.0
network/neural_network.h
Feed-forward network built from Node → Layer → Network classes. Any number of layers and nodes, a bias node per layer, tanh activation, mutation, and weight import/export.625Works; used by every trainer
Trainer v3.0
trainer_3_0/
Evolves 100 networks to balance a pendulum on a cart. Logs every network's fitness each round, replays the best network, plots with gnuplot.503Works
Trainer v2.1
old_trainer/
Evolves networks to fit one of 8 target functions from random samples, with input/output scaling and squared-error fitness.852Works
Multi-armed bandit
multi_armed_bandit/
A casino of slot machines with hidden payouts, and an ε-greedy player that learns which slot pays best.227Works after a one-character fix
Grid world + Q-learner
gridspace/, q_learner/
A 2D grid game where a human or bot player moves N/E/S/W to collect targets, with score and levels, plus the start of a tabular Q-learning agent.559 + 73Grid written; Q-learner unfinished
Agent–environment system 1.1
system/source/
General framework: abstract agent and domain interfaces, a system that runs the loop, a pluggable evolutionary algorithm, multi-agent support.~1,200Mid-refactor; doesn't compile as committed
Qt desktop GUI
system/_gui/
Configure and start runs, pause, export, and watch best and average fitness; a Data tab plots results.307 + layoutBuilt in Qt Creator

How it learns

Instead of backpropagation, the network is trained by evolution. The cart task has no correct action to learn from — only a cost for how far the pendulum is from upright — so evolution, which needs nothing but a fitness number, fits the problem.

  1. Network. Each layer holds nodes; every non-output layer adds a bias node that always outputs 1. A node sums its weighted inputs and applies tanh. Weights start uniform in [−1, 1].
  2. Mutation. Each weight changes with probability mutate_chance. The change is U(0, m) − U(0, m) for step size m, which makes small changes more likely than large ones.
  3. Evaluation. Every network in a population of 100 runs the task: 200 simulation steps of the cart, or 60 random inputs per round for function fitting. Lower average cost wins.
  4. Selection. Binary tournaments: pick two networks at random and keep the better one, until half the population survives.
  5. Reproduction. Refill the population with mutated copies of random survivors, then repeat — 1,000 rounds for the cart, 10,000 for functions.
  6. Output. The best network ever seen is replayed, and its states and the population's fitness history are written to CSV for plotting.

Results

All three runnable parts still do what they were built to do. Every run used my original settings and the code on main; the few changes needed to run it are listed under How the results were made.

Cart balance (trainer v3.0)

A 3-4-2 network (inputs cos θ, sin θ, θ̇; outputs torque and push force). Population 100, 1,000 rounds, mutation chance 0.1, step 0.2. Each test starts the pendulum 5° from upright and runs 200 steps of 0.1 s (20 s). Five independent runs, about 5.5 minutes each.

RunBest cost, round 0Best cost, round 1,000Time to within 0.01°Cart position at 20 sResult
165.60.0610.4 s2.8 mBalanced
2137.00.0450.3 s2.6 mBalanced
3167.143.6never245 mFailed: spun and drove off the track
4137.70.1571.1 s4.3 mBalanced
539.70.0570.6 s2.7 mBalanced

In the four successful runs the best network's cost fell by a factor of 700–3,000, and once within 0.01° the pendulum stayed there for the rest of the test. The cart stayed within its ±5 m track even though the network can't see the cart and nothing in the score rewards staying on it. Run 3 got stuck early in a poor solution — with tournament selection and no elitism, one run in five failing is plausible.

Best cost per round on a log scale: runs 1, 2, 4 and 5 fall to between 0.045 and 0.157; run 3 stalls at 43.6
Training. Best cost every 10th round of 1,000, five runs.
Pendulum angle over 20 seconds for each run's best network, starting 5 degrees from upright
Replay. Each run's best network flown from a 5° tilt, 200 steps of 0.1 s.

Function fitting (trainer v2.1)

A 1-4-1 network, population 100, 10,000 rounds of 60 random inputs on [0, 2], mutation chance 0.3, step 0.1. Each network was then measured on a 101-point grid it never trained on.

The network learned 7 of 8 functions closely: R² 0.96–0.999, and the constant exactly. cos x fails (R² 0.50) for a reason in the design, not the learning: the trainer scales outputs by f(xmax), and cos 2 is negative, so the targets fall outside tanh's range. The same scaling flattens sin x near its peak, where it exceeds sin 2.

Eight small charts comparing each target function with the best network's output on 0 to 2
Function fits. Best network of each 10,000-round run, sampled at 51 points on [0, 2]; R² from 101 points.

Multi-armed bandit (ε-greedy player)

1,000 casinos, each with 10 slots paying a hidden 0–9. The player pulls 100 times: 10 random pulls, then 60% greedy and 40% random, updating each slot's estimate with a running average.

MeasureResult
Best slot named by the player's estimates at the end94.6% of games
Pulls 51–100 spent on a best slot59.8%
Reward earned76.4% of optimal (random play: 52.9%)

The 59.8% is close to the ceiling the strategy allows, because it keeps exploring 40% of the time. A decaying exploration rate would close most of the remaining gap.

Code review

Looking back at it now: the core ideas are sound and the structure is clean for a first from-scratch ML project. The weaknesses are the usual ones of early research code — reproducibility, evaluation, and finishing the refactor.

What holds up

No libraries

Forward pass, activation, mutation, selection and scaling are all written by hand.

Clean layering

Node → Layer → Network, any topology from a nodes_per_layer list; one class serves three different trainers.

Gradient-free learning where it fits

Evolution trains a controller for a task with no labelled answers and a non-differentiable score.

Defensive engineering

Compile-time DEBUG/VERBOSE/TEST flags, size-mismatch checks, a progress bar and run timing.

Experiment tooling

Fitness history for every network every round, a replay of the best network, CSV logs and gnuplot scripts.

Architecture thinking

System 1.1 separates agent, domain and improvement algorithm behind interfaces — the shape of frameworks like OpenAI Gym, which appeared around the same time.

Limitations, and what I'd do now

IssueEffectWhat I'd do now
No elitism: the best network can lose a tournament and disappearSlower progress; contributes to stalled runs like run 3Always carry the best network forward
srand(time(0)) and no seed settingRuns can't be reproduced or compared fairlySeed from the command line; log the seed
Trainer v3.0 prints 5 lines per simulation stepAbout 100 million lines per run; slows every runPrint a summary per round only
Trained and tested from one start angle, and the network can't see the cartUnknown generalization; the cart could drift off the trackTrain on several start angles; add cart position and velocity as inputs
Outputs scaled by f(xmax)Fails when the target isn't largest at xmax (cos x)Scale by the target's actual range, or use a linear output layer
Hard-coded absolute pathsBreaks on any other machineRelative paths or a settings file
Bias nodes get outgoing weights they ignore4 of the 30 weights in the 3-4-2 network do nothingSkip weights into bias nodes
System 1.1 left mid-refactorThe unified framework never compiled end to endFinish one vertical slice before generalizing
Bandit: a loop typo; rewards cast to integers; fixed payoutsGreedy choice was wrong as committed; an easier problem than a real banditFix the loop; use doubles; add random payouts

The biggest lesson: bugs in learning code hide well. A selection bug can still look like slow learning, so test each stage on its own, log everything, and make runs reproducible from day one.

How the results were made

Re-run on November 17, 2026 from the main branch, compiled with g++ 15.2 (-std=c++11). The training and network code ran as committed, apart from these changes, made only in a scratch copy and never in the repository:

Selection fix. As committed, trainer v3.0 reset each network's fitness before selection, so selection was effectively random. Moving one line fixes it, and every cart-balance result above uses the fixed version.

  • Cart state log path. Pointed from an absolute path on my old machine to ./outputs/.
  • Function-fitting harness. Trainer v2.1 only logs average error, so I added two read-only accessors (best network, best error) and a small eval_main.cpp that trains exactly as main.cpp does, then samples the best network on a 101-point grid.
  • Trainer v2.1's network. Built against the v3.0 network it was written for, since the header it includes isn't next to it in the repository.
  • Bandit typo. i < current_standings.at(i) became i < current_standings.size(); a harness ran 1,000 games with the original Player class (seed 12345).
  • Not run: system 1.1 (doesn't compile mid-refactor), the Qt GUI (needs Qt Creator), the grid game (no main), the Q-learner (unfinished).

Trainer v3.0 seeds from the clock, so its five runs were started two seconds apart to get different seeds; by design of the original code they are not bit-for-bit reproducible.

cd agents/neural_network/trainer_3_0
make
mkdir -p outputs && ./run > /dev/null   # ~5.5 min; writes outputs/fitness_history.csv and pend_state_log.csv
wjhdfnmf/ProjectZeroC++ · main branch
View the code on GitHub

Next project: space_state — an N-body spacecraft trajectory simulator →