Skip to content

feat: context-sensitive completions — cursor-aware filtering & type suggestions#530

Merged
tnaum-ms merged 27 commits intofeature/shell-integrationfrom
dev/tnaum/context-sensitive-completions
Mar 18, 2026
Merged

feat: context-sensitive completions — cursor-aware filtering & type suggestions#530
tnaum-ms merged 27 commits intofeature/shell-integrationfrom
dev/tnaum/context-sensitive-completions

Conversation

@tnaum-ms
Copy link
Collaborator

@tnaum-ms tnaum-ms commented Mar 16, 2026

Step 4.5: Context-Sensitive Completions

Adds cursor-position-aware completion filtering to the documentdb-query language. Instead of showing all operators at every cursor position, completions are filtered based on the semantic context of where the cursor sits in the query expression.

Changes

Cursor context detection (cursorContext.ts)

  • Heuristic backward-scanning algorithm detects cursor position: key, value, operator, or array-element
  • Pure function — handles incomplete/mid-typing input gracefully
  • Field type enrichment via completion store lookup

Context-sensitive filtering

Cursor position Example Completions shown
Key { | } Field names + logical operators ($and, $or, $nor)
Value { age: | } Type suggestions + operators + BSON constructors + JS globals
Operator { age: { | } } Comparison operators ($gt, $eq, $in...) with stripped braces
Array element { $and: [ | ] } Same as key position
Unknown/empty Empty input Field names + key-position operators

Type-aware value suggestions (typeSuggestions.ts)

When the field BSON type is known from the schema, contextual suggestions appear first (sort prefix 00_):

  • booleantrue, false
  • number{ $gt: …, $lt: … } range query, { $gte: … }
  • string{ $regex: /…/ }, { $regex: /\.com$/ }, ""
  • dateISODate("…"), date range, { $gt: new Date(Date.now() - …) }
  • objectidObjectId("…")
  • array{ $elemMatch: { … } }, { $size: … }
  • nullnull

Type-aware operator sorting

When the field type is known, operators are sorted by relevance:

  • 0_ — Type-relevant (operator's applicableBsonTypes matches the field)
  • 1a_ — Comparison operators (universal, most commonly used)
  • 1b_ — Other universal operators ($exists, $type, $mod)
  • 2_ — Non-matching operators (demoted, not hidden)
  • 3_ — BSON constructors (ObjectId, ISODate)
  • 4_ — JS globals (Date, Math, RegExp, Infinity)

JavaScript global completions (jsGlobals.ts)

Restored JS globals that were lost when the custom documentdb-query language replaced the built-in Monaco JS worker:

  • Constructors: Date (→ new Date($1)), RegExp
  • Static methods: Date.now(), Math.floor(), Math.ceil(), Math.round(), Math.min(), Math.max()
  • Primitives: Infinity, NaN, undefined

Source: @mongodb-js/shell-bson-parser sandbox scope (the same runtime that executes queries).

Completion knowledge configuration (completionKnowledge.ts)

Centralised domain rules for the completion provider:

  • KEY_POSITION_OPERATORS — operators valid only at root/key position ($and, $or, $nor, etc.)
  • LABEL_PLACEHOLDER — the character for completion labels
  • INFO_INDICATOR — the character for descriptions

Field completion improvements

  • Selecting a field name now inserts fieldName: | (snippet with cursor at value position)
  • Field type shown right-aligned in the completion list via label.description

Validator enhancements

  • Unknown function calls → red error: Unknown function 'XyzAbc'. Expected a BSON constructor or a known global.
  • Unknown new constructors → red error: Unknown constructor 'Daddddte'.
  • Near-miss typos → yellow warning (unchanged): Did you mean 'ObjectId'?
  • Previously, unknown identifiers with Levenshtein distance > 2 produced no diagnostic at all

Projection/sort fix

  • PROJECTION_COMPLETION_META was ['field:identifier'] returning 0 static entries — projection operators ($slice, $elemMatch, $) and BSON constructors were never shown in project/sort editors. Fixed by adding 'query:projection' and 'bson' to the preset.

Bug fixes

  • Fix $gt snippet producing { : value }$ was interpreted as a Monaco snippet variable
  • Fix double-brace insertion for $and/$or at key position
  • Fix ISODate snippet using format pattern instead of example date

Refactoring

Extracted monolithic documentdbQueryCompletionProvider.ts into completions/ folder:

  • createCompletionItems.ts — main entry, context branching
  • mapCompletionItems.ts — operator/field → CompletionItem mapping
  • typeSuggestions.ts — type-aware value suggestions
  • jsGlobals.ts — JS globals from shell-bson-parser sandbox
  • completionKnowledge.ts — curated domain rules
  • snippetUtils.ts — snippet text manipulation
  • README.md — architecture and sorting documentation

Tests

  • 202 tests across 5 test suites — all pass
  • 41 cursor context detection tests (complete + incomplete input)
  • 95 completion provider tests (context-sensitive filtering, type sorting, field mapping)
  • 38 validator tests (syntax errors, typos, unknown identifiers, NewExpression)
  • 25 getFilteredCompletions tests
  • 3 completion store tests

Implements detectCursorContext() - a pure function that determines the
semantic position of the cursor within a DocumentDB query expression.

Supports: key, value, operator, array-element, and unknown positions.
Uses backward character scanning (no AST parsing) for robustness with
incomplete/mid-typing input.

Includes 41 tests covering:
- Core context detection (complete expressions)
- Incomplete/broken input (Step 1.5 from plan)
- Multi-line expressions
- fieldLookup integration for BSON type enrichment
Wire CursorContext into createCompletionItems() to filter completions
based on the semantic cursor position:

- key position: field names + key-position operators (, , etc.)
- value position: BSON constructors only (ObjectId, UUID, ISODate)
- operator position: comparison/element/array operators with type-aware sort
- array-element: same as key position
- unknown/undefined: full list (backward compatible)

All 48 existing tests pass unchanged. 16 new context-sensitive tests added.
Call detectCursorContext() in provideCompletionItems to detect the
semantic cursor position from editor text + offset. Pass the detected
CursorContext through to createCompletionItems() for context-sensitive
filtering.

Includes field type lookup via completionStore to enrich context with
BSON types for type-aware operator sorting.
Issue A - Strip outer braces from operator snippets at operator position:
  { $gt: ${1:value} } → $gt: ${1:value}  (when inside { field: { | } })
  Prevents double-brace insertion bug.

Issue B - Add category labels to completion items:
  Uses CompletionItemLabel.detail for operator category (comparison, logical, etc.)
  Uses CompletionItemLabel.description for operator description text.

Issue C - Value position now shows operators + BSON constructors:
  Operators sorted first (0_), BSON constructors second (1_).
  Operators keep their full brace-wrapping snippets at value position.

Also: trace-level console.debug logging in provideCompletionItems for debugging.

178 tests pass across 6 test suites. Build + lint clean.
- Remove 'detail' field from operator completions (was noisy)
- Move category label to label.description (matches field items pattern)
- Move entry.description to documentation (was in both detail and desc)
- Combine description text + docs link in documentation field
- Add range/insertText to trace logging for snippet debugging
Root cause: Monaco snippet syntax interprets $gt as a variable
reference (variable 'gt'), which resolves to empty string for unknown
variables. Result: { $gt: ${1:value} } produces '{ : value }'.

Fix: escapeSnippetDollars() escapes $ before letters ($gt → \$gt)
while preserving tab stop syntax (${1:value} unchanged).
BSON constructors like ObjectId("${1:hex}") are unaffected.

This was the root cause of POC Observation 2 (T5 ⚠️ PARTIAL).
…estions

Refactor:
- Extract completion logic into completions/ folder:
  - snippetUtils.ts: stripOuterBraces, escapeSnippetDollars
  - mapCompletionItems.ts: operator/field → CompletionItem mapping
  - typeSuggestions.ts: type-aware value suggestions (NEW)
  - createCompletionItems.ts: main entry point with context branching
  - index.ts: barrel exports
- documentdbQueryCompletionProvider.ts now re-exports from completions/

New feature — type-aware value suggestions at value position:
- bool fields → true, false (as first completions)
- int/double/long/decimal → range query { $gt: ▪, $lt: ▪ }
- string → regex snippet, empty string literal
- date → ISODate constructor, date range snippet
- objectid → ObjectId constructor
- null → null literal
- array → $elemMatch, $size snippets

Suggestions get sort prefix 00_ (before operators at 0_).
First suggestion is preselected.

197 tests pass across 6 suites. Build + lint clean.
The schema analyzer uses BSONTypes enum strings ('boolean', 'int32',
'decimal128') while the type suggestions used shortened names ('bool',
'int', 'decimal'). Fix the TYPE_SUGGESTIONS keys to match:
- bool → boolean
- int → int32
- decimal → decimal128
- Added: number (generic BSONTypes.Number)

Also noted the type-name mismatch between schema-analyzer BSONTypes
('int32') and documentdb-constants applicableBsonTypes ('int') —
tracked as a future normalization task.
Replace format pattern 'yyyy-MM-ddTHH:mm:ssZ' with a valid example
'2025-01-01T00:00:00Z'. The format pattern was confusing — users
typed over it with values like '2222-11-11T11:11:111' which fail
ISODate validation.

Also update date range snippet with realistic start/end dates.
@tnaum-ms tnaum-ms changed the title [WIP] context sensitive completions feat: context-sensitive completions — cursor-aware filtering & type suggestions Mar 16, 2026
tnaum-ms added 13 commits March 17, 2026 11:01
Previously `new Daddddte()` was not flagged because the validator only
handled CallExpression nodes. NewExpression has the same callee shape
but was not visited. Now unknown constructors in `new Foo()` produce
error diagnostics (red squiggles), and near-miss typos like `new Dae()`
produce warnings.
Add Date.now(), Math.floor(), Math.ceil(), Math.round(), Math.min(),
Math.max() as individual completion items alongside the class-level
Date/Math entries. Removed the bare `Math` entry since the individual
methods are more useful. All entries use sort prefix 4_ (after BSON
constructors at 3_).
…_COMPLETION_META

PROJECTION_COMPLETION_META was ['field:identifier'] which returned 0 static
entries — projection operators ($slice, $elemMatch, $) and BSON constructors
were never shown in project/sort editors. Added 'query:projection' and 'bson'
to the meta preset.
Never use git add -f to override .gitignore. Files in docs/plan/ and
docs/analysis/ are local planning documents excluded from the repo.
- Remove unnecessary escape in template literal (no-useless-escape)
- Formatter-applied changes to imports and whitespace
@tnaum-ms tnaum-ms marked this pull request as ready for review March 17, 2026 15:35
@tnaum-ms tnaum-ms requested a review from a team as a code owner March 17, 2026 15:35
@tnaum-ms tnaum-ms requested review from Copilot March 17, 2026 15:35
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances the documentdb-query Monaco language by making completions cursor-context-aware (key/value/operator/array-element) and by adding type-aware value suggestions and JS-global completions, plus associated refactoring and test coverage.

Changes:

  • Add heuristic cursor context detection (detectCursorContext) and wire it into the Monaco completion provider to filter/sort suggestions by cursor position.
  • Introduce type-aware value suggestions and restore JS global completions used by the @mongodb-js/shell-bson-parser sandbox.
  • Fix projection/sort completion presets in documentdb-constants and refactor completion logic into a dedicated completions/ module with extensive new tests.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/webviews/documentdbQuery/registerLanguage.ts Wires cursor context detection into Monaco completions; adds debug logging.
src/webviews/documentdbQuery/index.ts Re-exports cursor context utilities.
src/webviews/documentdbQuery/documentdbQueryValidator.ts Adds errors for unknown calls/constructors and supports NewExpression diagnostics.
src/webviews/documentdbQuery/documentdbQueryValidator.test.ts Adds/updates tests for unknown identifiers and constructors.
src/webviews/documentdbQuery/documentdbQueryCompletionProvider.ts Becomes a barrel re-export for the refactored completions module.
src/webviews/documentdbQuery/documentdbQueryCompletionProvider.test.ts Expands tests for context-sensitive completions, sorting, snippet escaping, and type suggestions.
src/webviews/documentdbQuery/cursorContext.ts New cursor context detector (heuristic backward scanning).
src/webviews/documentdbQuery/cursorContext.test.ts New unit tests for cursor context detection across complete/incomplete inputs.
src/webviews/documentdbQuery/completions/createCompletionItems.ts New context-sensitive completion routing and filtering logic.
src/webviews/documentdbQuery/completions/mapCompletionItems.ts New mapping utilities for operators/fields with sorting & snippet handling.
src/webviews/documentdbQuery/completions/typeSuggestions.ts New type-aware value suggestions shown at value position.
src/webviews/documentdbQuery/completions/jsGlobals.ts New JS global completion items (Date/Math/RegExp/etc.).
src/webviews/documentdbQuery/completions/snippetUtils.ts New utilities for brace stripping and $ escaping in Monaco snippets.
src/webviews/documentdbQuery/completions/completionKnowledge.ts New centralized completion rules/constants (key-only operators, placeholders).
src/webviews/documentdbQuery/completions/index.ts New module barrel exports.
src/webviews/documentdbQuery/completions/README.md Documents completion architecture and sorting tiers.
packages/documentdb-constants/src/metaTags.ts Fixes projection/sort preset to include projection operators and BSON constructors.
packages/documentdb-constants/src/getFilteredCompletions.test.ts Updates tests to validate the new projection preset behavior.
packages/documentdb-constants/src/bsonConstructors.ts Updates ISODate snippet placeholder to an example timestamp.
.github/copilot-instructions.md Adds git safety guidance (avoid git add -f, don’t commit planning docs).

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Tests were using 'int' but the schema analyzer produces 'int32' (BSONTypes.Int32).
Also fixed applicableBsonTypes in documentdb-constants bitwise operators to use
'int32' to match, which was a real production mismatch.
When cursorContext is undefined or 'unknown', show the full set of completions
(fields, all operators, BSON constructors, JS globals) instead of only key-position
items. This is a better UX fallback — when context can't be determined, the user
is better served by seeing everything available.
These debug logs were used during development and logged full editor text and
completion items to the webview console on every keystroke. Removed to avoid
unnecessary noise and potential data exposure.
@tnaum-ms tnaum-ms merged commit bab2efa into feature/shell-integration Mar 18, 2026
3 checks passed
@tnaum-ms tnaum-ms deleted the dev/tnaum/context-sensitive-completions branch March 18, 2026 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants