How I built a rubric auditor that runs in the browser

A university assignment arrives with two texts: the brief (what to hand in) and the rubric (how it is graded). The student finishes the document and is left with a fair doubt: did I cover everything the teacher asked for, in the order they asked for it? CrasterDoc tries to answer that before submitting, under a design constraint that shapes everything else: everything runs in the browser, and the part that reads the brief uses no AI.

This article walks through how that flow is built, with examples from the real code (src/lib/academico/). For the wider picture, see the page How CrasterDoc was built.

The contract: from loose text to a specification

The whole flow revolves around one type, EspecificacionActividad (the activity specification). It is what comes out of reading the brief, and what the auditor later uses to check the document:

interface Requisito {
  id: string // "ENT-1", "PASO-5", "C2", "FMT-FUENTE"…
  texto: string // what the teacher requires, in one sentence
  origen: 'paso' | 'entregable' | 'rubrica' | 'formato' | 'portada'
  criterio?: string // rubric criterion that scores it
  verificacion: 'automatico' | 'manual'
}

The key field is verificacion. Some requirements can be checked against the document text (there is a references section, the cover page has the group). Others can only be confirmed by the person (“the video is under five minutes”). That distinction runs through the rest of the design: what the program cannot check is marked manual and is never assumed to be met.

Reading a brief copied from Moodle

Briefs do not arrive clean: they are pasted from Moodle or a PDF, with inconsistent accents, long dashes, odd bullets. Regular expressions over that text are brittle, and normalizing it (lowercase, no accents) breaks the indexes you need to slice the original.

The fix is small and worth copying. normalizarTexto produces a copy with the same length as the original, character by character:

export function normalizarTexto(texto: string): string {
  let out = ''
  for (let i = 0; i < texto.length; i++) {
    const c = texto[i]
    if (/[‐-―−]/.test(c)) {
      out += '-' // every dash becomes "-"
      continue
    }
    const base = c.normalize('NFD')[0] ?? c // drops the accent
    const minus = base.toLowerCase()
    out += minus.length === 1 ? minus : base
  }
  return out
}

Because the copy has the same length, an index found in the normalized version works as-is to slice the original text and show the student the sentence with its accents and capitals. That small trick avoids keeping two position maps.

The rubric: three ways of saying the same thing

Every teacher writes the rubric differently. extraerRubrica tries three strategies in order: a pasted table (“Criterion | Excellent | Acceptable | Insufficient”), criteria with points per level, or levels per line. The result is a real scale, not an invented one:

export function escalaDeRubrica(rubrica: CriterioRubrica[]): EscalaRubrica | undefined {
  const total = rubrica.reduce((s, c) => s + c.maximo, 0)
  const conPeso = rubrica.every((c) => c.peso !== undefined)
  // …
  if (conPeso && Math.abs(total - 100) < 0.5) return { tipo: 'porcentaje', total }
  if (etiquetas) return { tipo: 'cualitativa', total, etiquetas }
  return { tipo: 'puntos', total }
}

Points, percentages, or qualitative levels (“Excellent”, “Good”…): the score the student sees uses the teacher’s scale, not a generic 0 to 100.

Tying each requirement to a criterion

A requirement such as “include references in APA format” has to end up under the right rubric criterion. There is no AI here: asociarCriterio compares word stems. It first reduces each word to an approximate stem (strips plurals and endings, cuts to seven letters), drops stop words and generic ones (“work”, “document”), and scores:

const p =
  3 * similitud(r, rc.nombre) + // matches the criterion name
  similitud(rs, rc.nombre) + // matches the section title
  Math.min(3, similitud(r, rc.descriptores)) // and what the levels say
if (p >= 3 && (!mejor || p > mejor.p)) mejor = { c, p }

If no criterion reaches the threshold of 3, the requirement stays unassigned. I would rather leave something loose than assign it wrongly.

The auditor: findings and an honest estimate

With the specification in hand, the auditor checks the document (or the text of an already exported PDF) and returns two things.

Findings with a severity. P0 is what should not ship as is (unfinished [PENDIENTE: …] placeholders, leftover template text); P1 and P2 are improvements. With an open P0 the result is bloqueante (blocking) and the PDF is exported with a DRAFT stamp.

One estimate per criterion. Each criterion collects the items that belong to it (sections present, complete cover page, well-formed references…) and averages them. Then aEscala maps that proportion to the rubric’s real level with a deliberately conservative rule:

// The top level only if everything verifiable is complete;
// if something is missing, at most the one below.
const tope = niveles.length - 1
const indice = p >= 0.999 ? tope : Math.min(tope - 1, Math.round(p * tope))

And, most importantly, when a criterion depends on something the program cannot judge, it returns nivelEstimado: null with the explanation “the teacher judges this”, and those points do not count. There are even explicit rules against self-deception: with no written section at all, neither the cover page nor the template’s formatting earns points; with no video link, a criterion that includes the video cannot go above 20 %.

The result is a score that is an estimate, labeled as such in the interface, and that can say “you are missing these three things for the top level”. That is more useful than a number that looks exact.

Why in the browser and without AI

• Privacy. A student’s brief and document never leave their machine. They only do if the person explicitly uses the optional AI assistant or the Drive copy.
• Determinism. The same brief always produces the same requirements, and that can be tested. The repository has synthetic brief fixtures (a history essay, a physics lab, a software project) and tests that pin down what the extractor must find.
• No per-query cost. There is no key to spend and no limit to explain for the core feature.

AI is kept for what really needs language (drafting), and it is optional.

What I learned

1. Model uncertainty as data. verificacion: "manual" and nivelEstimado: null force the interface to be honest; there is no way to forget them.
2. Small, testable heuristics beat one universal rule. Word stems, a threshold, a same-length normalized copy: each is explained in one line and has its test.
3. A conservative auditor builds trust. I would rather it say “not assessable” than celebrate a grade the teacher will not give.

CrasterDoc is free and needs no account: try it at crasterdoc.morelodev.com. To know who builds it, see About the author, and for the product itself, its page in this portfolio.