A Reproducible 10-Billion-Token Pretraining Run of GPT-2 124M
Technical report covering the training setup, evaluation and reproducibility of the Chris-GPT-2 pretraining run.

Software engineer · systems and machine learning
Software engineer working on business applications, machine-learning infrastructure and systems software. Professional development since 2012; independent work includes language-model training, inference runtimes, compiler tooling and operating-system development.
GitHub · LinkedIn · Hugging Face · Email · CV (PDF)
Production software. Enterprise management systems, fiscal and retail applications, desktop software, APIs and engineering tools.
Machine learning. Model pretraining, evaluation, GGUF inference, tensor computation and GPU-oriented execution.
Systems software. An x86-64 operating system, language toolchains, virtual machines and experiments with memory, drivers and graphics.
Experimental x86-64 operating system with kernel, memory-management, driver, graphics and filesystem development; also used for compiler and virtual-machine work.
C · C++ · x86-64 · OSDev · Compilers
LLM inference runtime in C: GGUF loading, tokenization, attention, KV cache, quantization and CPU/GPU execution.
C · GGUF · inference · quantization · CPU/GPU
Deep-learning implementation covering tensors, automatic differentiation, optimizers and accelerated execution backends.
C++ · Python · autograd · neural networks · GPU
Market-data and portfolio workspace for cross-asset tracking, risk metrics and portfolio-level analysis.
JavaScript · Cloudflare Workers · market data · risk analytics
Applied software projects spanning collaborative workspaces, live video, business applications, desktop utilities and online education.
Collaborative documents, tasks and workspaces with local persistence and synchronization between devices.
TypeScript · local-first · collaborative editing
Browser-based live production with video composition, visual overlays, audience interaction and commerce during broadcasts.
TypeScript · WebRTC · Canvas · media processing
Digital business cards and landing pages linked to lead collection, contact records and CRM operations.
Web applications · relational data · CRM
Disk-usage analysis and file cataloging with background scanning and native filesystem capabilities.
C# · .NET · Avalonia · filesystem APIs
Course administration, lessons, student progress and paid enrollment in a web application.
Next.js · TypeScript · database-backed workflows
Free guitar-learning platform with structured lessons, interactive chord diagrams, tablature playback, rhythm exercises and ear training.
Astro · TypeScript · Web Audio API · music theory
Technical report covering the training setup, evaluation and reproducibility of the Chris-GPT-2 pretraining run.
Study of continued pretraining and canonical instruction tuning for a Linux-focused GPT-2, including failure analysis.
I work on internal management and engineering systems covering finance, accounting, production planning and engineering. The stack includes C++, Delphi, MySQL, .NET/C#, React and PostgreSQL, with APIs, automated tests and architecture patterns such as DDD and CQRS.
Built web, mobile and desktop products for retail automation, fiscal documents, delivery, ticketing and messaging. Worked with PHP/Laravel, Angular, Vue, PostgreSQL, Java/Kotlin, React/Ionic, Delphi, Linux/Windows Server, AWS, DigitalOcean and Cloudflare.
Worked on retail automation, SAT fiscal processing, desktop update infrastructure, PHP/CodeIgniter APIs, PostgreSQL, Redis, internal tooling and cloud storage/querying of fiscal documents.
Started professionally with Delphi and later worked with PHP/MySQL, Python-based SMS tooling, GeneXus applications, R&D, AWS administration and OpenVPN.
Postgraduate study focused on electronics, robotics and autonomous systems.
Machine learning, deep learning, NLP, computer vision, big data, statistics, databases and software engineering.
In progress. Electrical engineering, mathematics, control, electronics and related engineering foundations.
Technical training across electronics, mechanics, automation and embedded systems.
Languages: C, C++, Python, Rust, C#, Java, Kotlin, JavaScript/TypeScript, PHP and Delphi.
Systems & ML: Linux, LLM training/inference, GPU computing, distributed training, operating systems, compilers, virtual machines, emulators and low-level debugging.
Application engineering: .NET, React, Laravel, REST APIs, PostgreSQL, MySQL, Redis, DDD, CQRS and automated testing.
Infrastructure: Git, GitHub Actions, Cloudflare, AWS, DigitalOcean and Linux/Windows Server.