---
language:
- code
tags:
- python
- java
- cpp
- ai-detection
- code-analysis
- temporal-cnn
- codet5
metrics:
- f1: 0.9921
---

# ai_code_detect

Binary classifier: human-written vs. AI-generated code. Trained on 500k samples (Python, Java, C++). Macro F1: **0.9813**.

---

## Architecture

Two input streams fused into a single MLP classifier.

**Stream 1 — Probabilistic**

Code is passed through `Salesforce/codegen-350M-mono`. Per-token surprisal signals are extracted across a 256-token window:

| # | Feature | Description |
|---|---------|-------------|
| 0 | `log_prob` | Log-probability of the actual token |
| 1 | `log_rank` | Log-rank within the distribution |
| 2 | `entropy` | Shannon entropy of the token distribution |
| 3 | `varentropy` | Variance of entropy |
| 4 | `top10_mass` | Probability mass in top-10 tokens |
| 5 | `gap_1_2` | Log-prob gap between rank-1 and rank-2 |
| 6 | `surprisal_z` | Per-token surprisal z-score |
| 7 | `entropy_delta` | Entropy change from previous position |
| 8 | `cum_rank` | Cumulative mean log-rank |
| 9 | `is_special` | Special token flag |
| 10 | `r10_flag` | Rank ≤ 10 |
| 11 | `r100_flag` | 10 < rank ≤ 100 |

These 12 per-token features aggregate into 32 sequence-level statistics (moments, autocorrelations, burstiness, etc.) passed downstream.

**Stream 2 — Semantic**

`Salesforce/codet5-base` mean-pools hidden states into a 768-dim embedding capturing style, structure, naming, and comment density.

**Classifier**

Token (256-dim) + sequence (64-dim) + semantic (768-dim) representations are concatenated → 1088-dim → 3-layer MLP with LayerNorm, GELU, dropout → sigmoid.

---

## Performance

Evaluated on 3,000 balanced validation samples (1,000/language):

| Metric | Score |
|--------|-------|
| Macro F1 | **0.9813** |
| Accuracy | **98.13%** |
| Threshold | 0.475 |

| Language | Accuracy | Human p̄ | AI p̄ | Gap |
|----------|----------|---------|-------|-----|
| Python | 99.50% | 0.001 | 0.992 | 0.991 |
| Java | 98.00% | 0.043 | 0.968 | 0.926 |
| C++ | 96.90% | 0.063 | 0.966 | 0.903 |

---

## Training

| Setting | Value |
|---------|-------|
| Optimizer | AdamW (encoder lr 8e-6, head lr 3e-5) |
| Scheduler | OneCycleLR + cosine annealing |
| Loss | BCEWithLogitsLoss |
| Regularization | EMA (decay=0.998), dropout, LayerNorm |
| Precision | fp16 via HuggingFace Accelerate |
| Hardware | 2× GPU |
| Epochs | 4 (500k samples) |


---

## How To Use

```python
import os
import sys
from huggingface_hub import hf_hub_download

REPO_ID = "santh-cpu/ai_code_detect"
script_path = hf_hub_download(repo_id=REPO_ID, filename="model.py")
sys.path.append(os.path.dirname(script_path))
from model import predict

print(predict("your code here"))
```