Cofos Code Model (v1-logic)

Cofos is a logic- and coding-oriented decoder-only language model built on AMEFORGE's SparseMind architecture. It is specialized for code generation, docstring construction, and code explanation in Python, JavaScript, and Rust.

Developed by AMEFORGE.

Architecture & Parameters

• Architecture: SparseMind v15 (dynamic neural type management, sparse attention, and sparse FFN)
• Parameters: 36,824,070
• Hidden dimension: 384
• Layers: 8
• Context length: 512 tokens
• Tokenizer: Custom BPE Tokenizer (vocab size 16384) with atomic indents/operators/keywords

Training Details

• Trained on 30,000,000+ procedurally generated programming templates on-the-fly.
• Masked loss to learn code completion and documentation formatting.
• Best Python syntax-validity on held-out validation: 10.8%

How to use

import torch
import sentencepiece as spm

# Load checkpoint
ckpt = torch.load("cofos_best.pt", map_location="cpu")
cfg_dict = ckpt["config"]

# Instantiate model architecture
# model = SparseMind(Config(**cfg_dict))
# model.load_state_dict(ckpt["model"])
# model.eval()