Building Finlang: A Language Server for Finacle Scripts

The problem was simple: Finacle scripts are hard to check until they run, and the gap between writing one and trusting one is uncomfortable.

A stray quote, an ENDIF that never arrives, or a repository variable that was never initialized all look plausible in an editor. The mistake only becomes obvious when the script finds its way into production, usually after a deploy, usually while someone is waiting for a banking transaction to complete. The runtime is less forgiving. The feedback loop has become expensive.

We built Finlang to bring that feedback into the editor. It provides live diagnostics, hover documentation, completion for Finacle keywords and urhk_* functions, and a limited amount of cross-file awareness for scripts invoked through CALL(...).

It is not a Finacle compiler. It is a practical language server: a set of checks we trust enough to keep turned on, backed by a protocol that lets the same engine work in more than one editor.

This is how we built it, what we changed our minds about and where we drew the line.

Why we didn’t write a Vim plugin

The obvious first move is an editor plugin. That is where the developer already is and it avoids inventing a protocol.

The problem is that language intelligence is the expensive part. Parsing, diagnostics, document state, symbol lookup, and documentation are not editor-specific, but a plugin makes them editor-specific anyway. Every editor then needs its own version of the same work. With enough editors and enough languages, you get an N × M maintenance problem.

The Language Server Protocol turns that into N + M. Each editor implements the client side once. Each language implements the server side once. Microsoft introduced LSP¹ in 2016, and it is now supported by VS Code, Neovim, Vim, Emacs, Sublime Text, and others.

For us, that was not an abstract architectural preference. It gave us four concrete things:

One place to encode language rules. The same diagnostics run in every editor, so a rule cannot drift between clients.
A shorter feedback loop. Errors appear while typing instead of after a deploy.
A testable boundary. The linter has a plain Python interface. We can test it without launching an editor.
Room to grow. We shipped diagnostics first, then hover and completion. None of those required rewriting the client integration.

We use Vim with vim-lsp² as our reference client, but the server does not know that. It speaks a standard protocol, which means the next editor is a configuration problem rather than a rewrite.

LSP in sixty seconds

An LSP server is a process that talks JSON-RPC 2.0 to an editor client. In our case the transport is standard input and standard output.

The client (the editor) sends two kinds of messages:

requests, which have an id and expect a response;
notifications, which have no id and expect no response.

The server can also send notifications. Diagnostics are the important example. The client does not wait for them; the server pushes textDocument/publishDiagnostics whenever it has a fresh result.

A session begins with initialize. The response is a contract: it tells the client which features the server supports. If we advertise hover, the client sends hover requests. If we advertise completion, it asks for completions. If we advertise formatting and never implement it, the editor exposes a command that does nothing.

Our capabilities are deliberately narrow:

{
  "capabilities": {
    "textDocumentSync": {
      "openClose": true,
      "change": 1
    },
    "hoverProvider": true,
    "completionProvider": {
      "resolveProvider": false,
      "triggerCharacters": ["_", "."]
    }
  },
  "serverInfo": {
    "name": "finlang",
    "version": "1.4.0"
  }
}

change: 1 means full document sync. Every change notification contains the entire buffer. That is unfashionable, but Finacle scripts are small enough that sending the whole document is cheap, and full sync removes a class of incremental-edit bugs we did not want to own.

The server supports initialize, textDocument/didOpen, textDocument/didChange, textDocument/didClose, textDocument/hover, textDocument/completion, shutdown, and exit. Unknown requests get a standard -32601 “Method not found” response. Unknown notifications are ignored.

The language we are serving

Finacle scripts are not a general-purpose programming language. They are a domain-specific runtime language for banking customization. They were not designed around a grammar we can hand to a parser generator, and most of its behavior is resolved at runtime.

A small example:

<--START

IF (FIELDEXISTS(CUST.DTLS.custPAN) == 1) THEN
#{
    CUST.DTLS.userId = BANCS.STDIN.userId
    sv_a = urhk_SetOrbWarning("Customer PAN is mandatory")
#}
ENDIF

END-->

A few things matter for tooling:

IF/ENDIF and WHILE/DO have their own pairing rules.
<--START and END--> are mandatory envelope tags.
sv_* variables are scratchpad variables; lv_* variables are local to a script.
CUST.DTLS.custPAN is a repository variable. Repositories and classes are created and deleted at runtime with CREATEREP, CREATECLASS, DELETEREP, and DELETECLASS.
CALL("setup") and CALLSCRIPTIFEXIST("setup") invoke other scr files.
Double-quoted strings do not treat a backslash as an escape character. In sv_a = "\", the final quote closes the string.
urhk_* functions are runtime helpers documented outside the language itself.

There is no formal grammar we can feed into a parser generator, and with so much behavior determined at runtime, we chose to build a tolerant analyzer rather than a strict parser.

Architecture

The architecture is compact enough to explain in a few lines:

Vim + vim-lsp (or another client)
        │  JSON-RPC 2.0 over stdio
        ▼
      main.py         logging, stdin reader thread, event loop, lifecycle
        ▼
      server.py       documents, routing, diagnostics scheduling, hover, completion
        ▼
      linter/         shared lexer, rules, type checker, cross-file scan
        ▼
  workspace files     CALL(...) scripts, finacle_docs.json

Path	Responsibility
`main.py`	Process startup, logging, stdin reader thread, event loop, lifecycle
`server.py`	JSON-RPC framing, document state, routing, debounce, hover, completion
`linter/__init__.py`	Rule registry and diagnostics engine
`linter/text.py`	Shared lexer producing raw, code, and masked views
`linter/finacle.py`	Shared patterns for calls, repositories, classes, assignments
`linter/typecheck.py`	Tokenizer, expression parser, type state
`linter/rules/`	One module per diagnostic rule
`finacle_docs.json`	Hover and completion documentation
`vimrc`	Reference vim-lsp configuration
`test_*.py`	Unit and regression tests

Separating process management into main.py and protocol logic into server.py keeps both easier to test. The server module does not start threads or configure logging. The entry point does not know how a hover request is answered.

The wire: JSON-RPC framing

LSP over stdio is framed with a Content-Length header:

Content-Length: 123\r\n
\r\n
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{...}}

The length is in bytes, not characters. Getting that wrong is a classic way to produce a server that works until someone types a non-ASCII character.

The reader consumes header lines until a blank line, parses Content-Length, and reads exactly that many bytes. We cap the body at 64 MiB. A malformed or hostile length should not make the server allocate an unbounded buffer. Truncated bodies are rejected rather than parsed.

On the way out we use compact JSON and serialize writes with an asyncio.Lock:

body = json.dumps(payload, separators=(",", ":")).encode("utf-8")
header = f"Content-Length: {len(body)}\r\n\r\n".encode("latin1")

async with _get_write_lock():
    sys.stdout.buffer.write(header)
    sys.stdout.buffer.write(body)
    sys.stdout.buffer.flush()

The lock is not optional. Several request handlers can finish at once, and interleaving two JSON bodies on stdout corrupts the stream in a way that is painful to diagnose.

The other rule is simple: nothing else writes to stdout.

Reading stdin without stalling the server

A language server must keep reading messages while it processes requests. If we block the event loop in sys.stdin.readline(), diagnostics and hover requests stop until the next message arrives.

Running that read in the default thread pool is also a bad fit: it occupies a worker forever. Instead, main.py starts a dedicated daemon thread whose only job is to read one framed message and hand it to an asyncio.Queue:

reader = threading.Thread(
    target=_reader_thread,
    args=(loop, queue),
    name="finlang-stdin",
    daemon=True,
)
reader.start()

The event loop then creates one task per message. Messages are read in order, but handlers can overlap. A slow lint does not prevent a hover from being answered, and the write lock keeps the wire well-formed.

Diagnostics run in a single-worker ThreadPoolExecutor:

LINT_POOL = ThreadPoolExecutor(
    max_workers=1,
    thread_name_prefix="finlang-lint",
)

That is deliberate. The work is CPU-bound Python. More workers would mostly compete for the GIL, increase memory pressure, and make “latest edit wins” harder to reason about. One worker keeps the policy simple.

The exit path is deliberately blunt. When the client sends exit, we flush the streams and call os._exit(0). The stdin reader is a daemon thread blocked in a buffered read, and a normal interpreter shutdown can abort with a fatal “buffered busy” error when a daemon thread is still inside a buffered I/O call. exit is an instruction to terminate now, so we flush and leave. The normal EOF path is graceful: pending tasks are cancelled, the executor is shut down, and the server logs that it stopped.

Diagnostics: latest edit wins

Diagnostics are the center of the server. Document state is intentionally smoll:

@dataclass
class Document:
    text: str
    version: int = 0
    revision: int = 0
    lint_task: Optional[asyncio.Task] = None

version is the client’s document version. revision is our own monotonic counter, shared across open and close cycles. The distinction matters because a document can be closed and reopened, so the client’s version is not enough to decide whether a lint result is still current.

The pipeline is:

Store the latest text and version.
Increment the revision.
Cancel the previous lint task if it is still running.
Start a new debounce task.

The debounce task waits, runs the linter in the worker pool, and publishes diagnostics:

await asyncio.sleep(DEBOUNCE_SECONDS)

document = DOCUMENTS.get(uri)
if document is None or document.revision != revision:
    return

diagnostics = await loop.run_in_executor(
    LINT_POOL, run_diagnostics, uri, document.text
)

document = DOCUMENTS.get(uri)
if document is None or document.revision != revision:
    return

await send_message({
    "jsonrpc": "2.0",
    "method": "textDocument/publishDiagnostics",
    "params": {
        "uri": uri,
        "version": document.version,
        "diagnostics": diagnostics,
    },
})

The default debounce is 150 ms, configurable with FINLANG_DEBOUNCE. The reference Vim configuration lowers it to 50 ms because the server is already coalescing edits, and vim-lsp’s own event queue would otherwise add another second of latency.

The two revision checks are not redundant. The first catches a change that arrived during the debounce sleep. The second catches a change that arrived while the linter was running. Without it, this sequence breaks:

The user types, starting a lint of revision 10.
The linter takes 80 ms.
The user types again, creating revision 11.
The revision 10 lint finishes and publishes stale diagnostics.

The result is a squiggle on code the user already fixed, or a diagnostic that disappeares because an older result arrived last.

On textDocument/didClose, we remove the document, cancel its pending lint, and publish an empty diagnostics list so the client clears anything left behind.

Full sync keeps the change handler small. The last entry in contentChanges carries the complete document, so we assign it rather than applying ranges:

changes = params.get("contentChanges") or []
if changes:
    last_change = changes[-1]
    if "text" in last_change:
        document.text = last_change["text"]

More bytes on the wire, fewer edit-reconciliation bugs on our side. For the size of a typical Finacle script, that is the right trade.

The linter: a set of small opinions

linter/__init__.py owns the rule registry:

ACTIVE_RULES = (
    UnclosedQuotesRule,
    BlockMatchingRule,
    EnvelopeTagsRule,
    IfSyntaxRule,
    UninitializedVariableRule,
    VariableTypeRule,
)

Every rule has the same interface:

@classmethod
def analyze(cls, uri, text, lines, views=None) -> list[dict]:
    ...

The engine builds shared views once and passes them to every rule. If a rule raises, we log it and keep going. One broken rule should not blank diagnostics for the whole file.

One lexer, three views

Finacle’s comment and string rules are subtle enough that we did not want each rule to implement its own scanner. linter/text.py computes three views per line in one pass:

View	Contents	Used for
`raw`	The original line, including comments and strings	Reporting full-line ranges
`code`	The line with `#` comments removed, but `#{` / `#}` preserved	Structural checks
`masked`	`code` with string literals replaced by spaces	Keyword and identifier scanning

Strings are replaced with spaces rather than removed, so columns still line up with the original text. Comments are truncated, not rewritten, for the same reason.

The lexer also records whether a quote was left open:

@dataclass(frozen=True)
class LineView:
    raw: str
    code: str
    masked: str
    unclosed_quote: Optional[str] = None

Some language-specific decisions live here:

strings match before comments, so # inside a string is not a comment;
double-quoted strings end at the next double quote, because a backslash is an ordinary character;
single-quoted strings do keep backslash handling;
an unterminated literal is recorded, and the rest of the line is masked so one missing quote does not cascade into a second false string.

Sharing the lexer was one of the better decisions in the project. Before it, several rules had their own comment/string state machines and disagreed at the edges. Now they all see the same line.

Hover and completion

Hover is a small feature with a large onboarding payoff. Put the cursor on urhk_dbSelectWithBind, press K, and get the documentation without leaving Vim.

The documentation lives in finacle_docs.json. At startup we load it into a case-insensitive lookup table.

The response is markdown:

### Finacle Reference

`urhk_dbSelectWithBind`

**urhk_dbSelectWithBind(query)**: Selects data from the database using bind variables.

If no documentation exists, the server still returns the symbol name in a generic hover. Less useful, but better than an empty popup.

Completion is precomputed at startup from the same documentation file. Each item has a label, a filterText with the original spelling, an LSP kind, a detail string, and markdown documentation.

resolveProvider is false because every item already includes its documentation. Completion currently returns the full list and lets the client filter it. That is fine for a few dozen items. If the documentation grows into the thousands, we will need server-side filtering and ranking.

The Vim side

The server is editor-agnostic. Our reference client is Vim with vim-lsp, and the repository includes a vimrc that can be sourced from an existing config or used standalone.

It does three things:

Registers the server for the finacle filetype.
Turns on the diagnostics UI we want: virtual text below the line, signs, and highlights, including in insert mode.
Disables features the server does not implement, so the editor does not generate unsupported requests.

The registration is small:

call lsp#register_server({
      \ 'name': 'finlang',
      \ 'cmd': {server_info -> [g:finlang_python, g:finlang_server]},
      \ 'allowlist': ['finacle'],
      \ 'root_uri': function('s:finlang_root_uri'),
      \ })

Two settings are worth calling out:

let g:lsp_use_event_queue = 0
let $FINLANG_DEBOUNCE = '0.05'

vim-lsp normally waits before forwarding changes. Since the server already debounces and supersedes stale work, we disable that queue and let the server own the timing. The 50 ms debounce keeps typing responsive without linting every intermediate keystroke.

Buffer-local mappings make the features discoverable: <C-Space> opens completion, K shows hover, [g and ]g move between diagnostics, <leader>d opens the diagnostic list, and <leader>ls shows server status.

To try it from an existing Vim configuration:

source ~/Developer/finlang/vimrc

Or start a clean Vim with only the Finlang configuration:

vim -u ~/Developer/finlang/vimrc path/to/script.scr

The server writes to a rotating log in the system temporary directory by default:

tail -f "${TMPDIR:-/tmp}/finlang_lsp.log"

Set FINLANG_LOG_DIR before launching Vim if you want the log somewhere more permanent.

A few things we’d do differently

Two of our early decisions still show up in the code comments.

The first was letting each rule do its own comment and string scanning. It felt local and simple: a rule needed to know whether a # was inside a string, so it handled that itself. The rules slowly drifted. A quote inside a comment was handled one way in one rule and another way somewhere else. We eventually pulled that into linter/text.py, which now produces the raw, code, and masked views once. New rules got shorter, and the diagnostics stopped contradicting each other.

The second was scanning block markers character by character inside BlockMatchingRule. That worked, but it duplicated logic the lexer already had and made the rule harder to reason about. The current version walks the masked line with a regular expression and three stacks. It is not clever, but it is easier to test.

The third is less concrete. We spent too long treating “complete” as the goal. A language server does not need to understand every Finacle construct to be useful; it needs to be right about the constructs it does understand. Most of the value came from a handful of checks that are boring, local and reliable.

We still have to resist that instinct when adding a rule. The question is not “can we detect this?” It is “will a developer thank us for this squiggle?”

What is missing

Finlang is deliberately scoped. It does not support go-to-definition, find references, rename, formatting, code actions, signature help, or semantic tokens. It does not do incremental document sync. It does not maintain a full workspace index or recursively walk the call graph. It cannot resolve dynamically constructed script paths. It does not implement a complete Finacle grammar.

The next steps are obvious enough: incremental sync for very large files, a workspace index for definitions and references, quick fixes for common diagnostics such as missing #}, ENDIF, or envelope tags, and signature help for the urhk_* functions. None of those require rethinking the architecture. The editor integration stays stable while the language intelligence improves.

Where this leaves us

Finlang started as a linter and became a language server because the protocol gave us a clean boundary between what we know about Finacle and how the editor displays it.

The implementation is small enough to hold in your head:

main.py handles the process and the transport;
server.py handles documents, routing, diagnostics, hover, and completion;
a shared lexer gives every rule the same view of a line;
a handful of focused rules encode what we know about the language;
a conservative type checker avoids guessing;
a one-hop cross-file scan follows CALL(...);
and a Vim configuration turns it into a daily tool.

It is not a compiler, and it is not trying to be. It is a set of checks we trust enough to leave on, with documentation where the language is obscure and diagnostics where the runtime is unforgiving.

The server is in day-to-day use. That is the only benchmark that matters to us.