Development

What is where, which command runs which gate, and the conventions a change is expected to follow.

Repository layoutCopy Anchor Link

Maljan/
├── apps/
│   ├── api/           FastAPI app, arq worker, Alembic migrations ("maljan-api")
│   ├── docs/          this documentation site: pages under content/docs/,
│   │                  images under public/assets/
│   └── web/           Next.js console, Playwright specs under e2e/
├── src/maljan/        the core package: agents, pipeline, analysis layers,
│                      providers, memory, reporting, core (config, container)
├── services/          stdio MCP sidecars, one server.py each
├── scripts/
│   ├── ci/            the uv.lock dependency-graph submission the workflow runs
│   ├── dev/           fetch_external.sh and the Ghidra image manager
│   ├── goldens/       capture scripts that write tests/fixtures/golden/, plus
│   │                  render_team_graphs.py, which draws the team-*.svg diagrams
│   └── knowledge/     builders for the data/ assets
├── tests/             unit/ api/ integration/ fixtures/
├── data/              tracked knowledge assets, loaded lazily, each with a fallback
├── docker/            Dockerfiles, the compose stack and its dev overlay
├── docs/              examples/: the importable all-tools team document
├── Makefile           every gate and every generator
└── pyproject.toml uv.lock   one uv workspace: maljan plus apps/api

The repository is one uv workspace with one lockfile. uv sync --all-extras --all-packages installs both Python packages, so import maljan and import app work from any directory without PYTHONPATH, in the venv, in the backend image and in CI alike.

Route-local React components stay in their route folder; a component two routes use moves to apps/web/src/components/. tests/unit/ mirrors src/maljan, one subdirectory per subpackage, and is where a new test starts.

Host-specific helpers (a launcher for a local llama-server, a memory guard, a restart wrapper for long evaluations) live outside the repository. For the record, the local model server the measurements in this repository were taken with ran ik_llama.cpp as llama-server -m Qwen3.6-35B-A3B-IQ3_K_R4.gguf -c 131072 -t 16 -fa on -ctk q8_0 -ctv q8_0 -ngl 999 -ot 'blk\.([1-3][0-9])\.ffn_(up|gate|down)_exps=CPU' --context-shift on --jinja on loopback port 8080.

Make targetsCopy Anchor Link

TestsCopy Anchor Link

• Python — uv run pytest tests/ -q. tests/unit/ is the default home; tests/api/ covers the FastAPI surface and tests/integration/ the flows that cross the worker, the database and object storage. The suite needs no secret in the environment: a pytest-only JWT secret is substituted, and the auth bypass is forced off so real 401 and 403 assertions still mean something.

• Goldens — scripts/goldens/capture_provider_goldens.py freezes the sandbox tools' answers per report under tests/fixtures/golden/sandbox_tools/. Most source reports (data/cape_reports/) are kept only on the machine that captured them; two are tracked, one of them a hand-written report (tests/fixtures/sandbox/cape2_synthetic.json) whose registry calls and network rows CI compares. A case whose source is absent is skipped, and the session ends with the number of such skips (on GitHub Actions also a warning annotation and a line in the job summary). MALJAN_TEST_NO_CAPE_REPORTS=1 takes that path on a machine that has the reports. A golden whose text passes 5 MB is written as <name>.json.gz and read back decompressed. The same script writes the two prompt fixtures under tests/fixtures/prompts/, which a test compares with the live prompt builder, so a prompt change comes with its recapture.

• Frontend unit — cd apps/web && npm run test:unit (vitest), alongside npx tsc --noEmit and npm run lint.

• Playwright — cd apps/web && npm run test:e2e. The suite starts its own next dev on port 3100, forces NEXT_PUBLIC_AUTH_DISABLED=false and points the client at the Next server's own origin, so it is hermetic and contacts no backend; every API call is mocked in e2e/mocks.ts. Run a named spec while developing (npx playwright test e2e/settings-configuration.spec.ts --project=chromium) rather than the whole suite, and stop any dev server you started yourself first — the browsers are memory-hungry, which is why the local worker count and the timeouts are raised deliberately.

• Replay — python scripts/replay/run.py <stored run> re-runs the platform's deterministic rules over a stored run with no model call, and --compare REF_A REF_B names every difference a rule change makes between two refs. Run it before and after a change to a check, the report builder or a publish rule; see Replaying a stored run.

The evaluation harness behind the published measurements is no longer part of this tree; see paper.md.

Native runs pin LC_CTYPECopy Anchor Link

C code folds case with the C library's tolower, by the process's LC_CTYPE. Under a Turkish or Azerbaijani locale a capital I does not fold, so the YARA engine's nocase misses every I it should match and its pe.imphash() differs. Each of the platform's own processes therefore sets LC_CTYPE, and nothing else, to C.UTF-8 (or C.utf8) when it starts, before any thread or the YARA engine loads, and keeps it only when the C library then folds I to i (core.ctype_locale.pin_character_classification). This covers the analysis worker, the maljan CLI, scripts/validation/run.py, scripts/rehearsal/run.py and the analysis and network sidecars, whose line goes to standard error because standard output is their protocol. The test session pins the same way in tests/conftest.py, and a test that changes the locale itself runs in a subprocess. Messages, times and collation stay the operator's. On a host whose locale is not UTF-8 (a legacy 8-bit one such as tr_TR.ISO-8859-9), the pin also makes UTF-8 Python's preferred encoding, which matches the image; the INFO line says so when the encoding changes. An operator's own tool server keeps the environment it was started with, and so the operator's locale: a YARA engine inside one can still mishandle I and i on such a host. The backend image already runs under LANG=C.UTF-8.

Documentation siteCopy Anchor Link

The pages are MDX files under apps/docs/content/docs/, built into a static site with Fumadocs on Next.js, and published to GitHub Pages from main by .github/workflows/docs.yml. The sidebar's order and groups are content/docs/meta.json. Preview it locally from apps/docs with npm ci and npm run dev; npm run build writes the static site to apps/docs/out/ and fails if any address the site has served before no longer leads to a page (scripts/old-urls.mjs).

Validation codes is generated, not written. Run uv run python scripts/validation_codes/run.py and commit the page after any of these edits:

• adding, removing or renaming a code, or the constant that names it;
• adding, moving or removing a place that raises a code or passes one on, renaming the function it is in included;
• changing a code constant's comment, the comment over a group of code constants, or the message a raise site passes;
• adding or removing a mention of a code on a docs page, or renaming or moving such a page;
• adding or changing a code position the register cannot follow.

Adding, changing or removing a test does not change the page: the tests are not listed on it, and run.py --tests prints which tests name each code. The register reads only the files git tracks, so add a new file before regenerating. tests/unit/test_the_validation_code_register_is_current.py fails when the committed page is not what the source gives, naming the command, when nothing raises a code, and when no test names one.

Continuous integrationCopy Anchor Link

.github/workflows/ci.yml runs on pushes to main and dev and on pull requests into main, dev or feat/**:

Three more workflows run beside CI. CodeQL analyses Python, TypeScript and the workflow files themselves on every push and pull request to main and dev, with the security-extended query suite, and weekly. Dependency review blocks a pull request that introduces a dependency with a known high or critical vulnerability or a copyleft licence. OpenSSF Scorecard runs on main weekly and publishes its findings to code scanning. Dependabot opens weekly update pull requests against dev for the uv workspace, the web console, the pinned GitHub Actions and the Docker base images; security updates arrive as soon as an advisory matches. Every action in a workflow is pinned to a commit with its version in a comment, which Dependabot keeps current. A dependency submission workflow posts the packages resolved in uv.lock to the dependency graph on every push to main, so Dependabot's Python alerts follow the lockfile the way the console's follow package-lock.json. Secret scanning with push protection is on for the repository, and vulnerabilities are reported through private vulnerability reporting (see SECURITY.md).

ConventionsCopy Anchor Link

• Conventional commits, one logical slice per commit.
• Feature branches start from dev and are merged into dev; main is branch-protected and only ever advanced by an explicit promotion pull request from dev.
• No question sentences in headings, comments or documentation.
• A comment explains why the code is the way it is. Process tags — dated audit identifiers, ticket numbers, "new in phase B" — do not belong in the tree; the reasoning stays, the bookkeeping goes.
• New settings need an entry in src/maljan/core/settings_annotations.py, or they appear in the console under their dotted path with no description.
• A removal is not done until grep shows zero remaining references across src apps tests scripts docs and the affected tests are updated.