OpenArch – Modern LLM mimarilerinin PyTorch uygulamaları

Python implementations of modern open-source LLM architectures — written from scratch, one model at a time.

This repository contains hand-written PyTorch implementations of the model architectures cataloged in Sebastian Raschka's LLM Architecture Gallery . Each model is implemented to the best of my knowledge from the original papers, technical reports, reference config.json files, and the excellent writeups by Sebastian Raschka and Machine Learning Mastery.

The goal is not to compete with transformers or other production libraries. The goal is clarity and learning : a single readable file per architecture, with the structural choices (attention type, normalization, layer mix, MoE routing, positional encoding) made explicit and easy to compare side-by-side.

Modern LLM architectures share a common skeleton but differ in dozens of small, important choices:

Reading the official model code can be hard because production repos optimize for speed, sharding, and backward compatibility. This repo optimizes for reading .

Implementations marked ✅ are usable for forward passes; those marked 🚧 are under construction.

The full target list mirrors the 72 architectures in the Architecture Gallery. Contributions toward any of them are welcome.

Each model lives in its own folder with respective model.py and a short README.md describing the architectural choices and references used.

I am actively looking for contributors.

If you enjoy reading model papers, comparing config.json files, or just want to deepen your understanding of how modern LLMs are built, this is a friendly place to start.

Please open an issue before starting a large piece of work so we can avoid duplicating effort. Implementations should prioritize readability over performance — this is a learning resource first.

This repository would not exist without the work of two outstanding educators:

This project is licensed under the Apache License 2.0 — see LICENSE for details. Individual model implementations follow the licenses of the original models where applicable; see each model's folder for specifics.

These implementations are written to the best of my knowledge based on publicly available papers, technical reports, configuration files, and educational material. They are intended as a learning resource and are not affiliated with or endorsed by the original model authors.