CorX3.8-27B is a conversational assistant that speaks Jamaican Patois by default and English on request. It is a fine-tune, not a model trained from scratch: a rank-64 LoRA adapter on every attention and feed-forward projection of the open-weight Qwen3.8-27B base, a 64-layer hybrid of linear and full attention, trained for one epoch of about 19 million tokens and merged into standalone weights. Training loss fell from 1.95 to 0.89 and next-token accuracy rose from 65% to 76%, nearly all of it in the first hundred steps. We describe what the fine-tune changed (voice, register and cultural grounding) and what it did not (the base model's reasoning and knowledge), why factual use needs the retrieval layer the lab ships, and the absence so far of any benchmark or Patois evaluation.
CorX3.8-27B is a conversational assistant that speaks Jamaican Patois by default and switches to standard English when the user writes in English or asks for it. It is not trained from scratch. It is a fine-tune of the open-weight Qwen3.8-27B base (Apache-2.0): the reasoning, code and mathematics come from that base, and CorX Labs added the Jamaican voice, a direct register and everyday cultural grounding. That makes it a different kind of achievement from CorX1.5, which was trained from random weights, and the lab does not present them as the same thing.
From the released config.json, the base is a 64-layer hybrid: three gated linear-attention layers for every
full-attention layer, so 48 linear and 16 full.
| Setting | Value |
|---|---|
| Hidden size / feed-forward size | 5,120 / 17,408 |
| Full attention | 24 query heads, 4 key-value heads, head size 256, output gate |
| Linear attention | 16 key heads and 48 value heads of 128, short convolution of width 4 |
| Vocabulary | 248,320 |
| Maximum positions | 262,144 |
| Weights | bfloat16, about 55 GB; about 24 GB of GPU memory in 4-bit |
The fine-tune is a LoRA [1] adapter trained on every attention and feed-forward projection (q, k, v, o, gate, up and
down), then merged into the base so that the released model needs no adapter step and no separate download. The settings
below are read from the adapter and trainer state shipped in the repository's last-checkpoint/ folder.
| Setting | Value |
|---|---|
| LoRA rank / alpha / dropout | 64 / 128 / 0.05 |
| Epochs / optimiser steps | 1 / 598 |
| Peak learning rate | 1e-4, decayed to under 1e-6 by the end |
| Tokens seen | about 19 million (18.8 million by step 575) |
| Training loss, step 1 to step 575 | 1.95 to 0.89 |
| Mean next-token accuracy, step 1 to step 575 | 65.2% to 76.4% |
Figure 1. Training loss from the shipped trainer state. Almost all of the change happens in the first 100 steps; from there the loss sits near 0.9 for the rest of the epoch. That is the shape expected when an adapter is learning a register and a voice rather than new knowledge, and it also says that more steps on the same data would buy little. There was no separate evaluation split in this run.
| Area | Lab's assessment |
|---|---|
| Jamaican Patois conversation | Strong |
| Direct, honest register | Strong |
| Common Jamaican cultural reference | Good |
| General reasoning, code and mathematics | Good, inherited from the base |
| Specific facts (roads, laws, prices) | Weak; use retrieval |
| Current information | None; no live data |
These are the lab's qualitative judgments from use, not benchmark scores. No standard benchmark or Patois evaluation set has been run.
The model has a Jamaican voice but is not a Jamaican knowledge base; it does not reliably know addresses, current laws, exam schedules or prices. The lab therefore ships a retrieval layer that grounds answers in OpenStreetMap geography and public government documents. With it attached, the model answers factual questions from cited material and says when something is not in its sources. This is the recommended setup for anything factual.
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("CorxLabs/CorX3.8-27B")
model = AutoModelForCausalLM.from_pretrained("CorxLabs/CorX3.8-27B", dtype="bfloat16", device_map="auto")
Merged weights under Apache-2.0, inherited from the base with attribution retained, at
huggingface.co/CorxLabs/CorX3.8-27B; it can also be tried at corx-labs.com/chat.