Unnamed repository; edit this file 'description' to name the repository.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
import * as vscode from 'vscode';
import * as os from "os";

import * as commands from './commands';
import { activateInlayHints } from './inlay_hints';
import { Ctx } from './ctx';
import { Config } from './config';
import { log, isValidExecutable, isRustDocument } from './util';
import { PersistentState } from './persistent_state';
import { activateTaskProvider } from './tasks';
import { setContextValue } from './util';
import { exec } from 'child_process';

let ctx: Ctx | undefined;

const RUST_PROJECT_CONTEXT_NAME = "inRustProject";

export async function activate(context: vscode.ExtensionContext) {
    // VS Code doesn't show a notification when an extension fails to activate
    // so we do it ourselves.
    await tryActivate(context).catch(err => {
        void vscode.window.showErrorMessage(`Cannot activate rust-analyzer: ${err.message}`);
        throw err;
    });
}

async function tryActivate(context: vscode.ExtensionContext) {
    const config = new Config(context);
    const state = new PersistentState(context.globalState);
    const serverPath = await bootstrap(context, config, state).catch(err => {
        let message = "bootstrap error. ";

        message += 'See the logs in "OUTPUT > Rust Analyzer Client" (should open automatically). ';
        message += 'To enable verbose logs use { "rust-analyzer.trace.extension": true }';

        log.error("Bootstrap error", err);
        throw new Error(message);
    });

    if ((vscode.workspace.workspaceFolders || []).length === 0) {
        const rustDocuments = vscode.workspace.textDocuments.filter(document => isRustDocument(document));
        if (rustDocuments.length > 0) {
            ctx = await Ctx.create(config, context, serverPath, { kind: 'Detached Files', files: rustDocuments });
        } else {
            throw new Error("no rust files are opened");
        }
    } else {
        // Note: we try to start the server before we activate type hints so that it
        // registers its `onDidChangeDocument` handler before us.
        //
        // This a horribly, horribly wrong way to deal with this problem.
        ctx = await Ctx.create(config, context, serverPath, { kind: "Workspace Folder" });
        ctx.pushCleanup(activateTaskProvider(ctx.config));
    }
    await initCommonContext(context, ctx);

    activateInlayHints(ctx);
    warnAboutExtensionConflicts();

    ctx.pushCleanup(configureLanguage());

    vscode.workspace.onDidChangeConfiguration(
        _ => ctx?.client?.sendNotification('workspace/didChangeConfiguration', { settings: "" }).catch(log.error),
        null,
        ctx.subscriptions,
    );
}

async function initCommonContext(context: vscode.ExtensionContext, ctx: Ctx) {
    // Register a "dumb" onEnter command for the case where server fails to
    // start.
    //
    // FIXME: refactor command registration code such that commands are
    // **always** registered, even if the server does not start. Use API like
    // this perhaps?
    //
    // ```TypeScript
    // registerCommand(
    //    factory: (Ctx) => ((Ctx) => any),
    //    fallback: () => any = () => vscode.window.showErrorMessage(
    //        "rust-analyzer is not available"
    //    ),
    // )
    const defaultOnEnter = vscode.commands.registerCommand(
        'rust-analyzer.onEnter',
        () => vscode.commands.executeCommand('default:type', { text: '\n' }),
    );
    context.subscriptions.push(defaultOnEnter);

    await setContextValue(RUST_PROJECT_CONTEXT_NAME, true);

    // Commands which invokes manually via command palette, shortcut, etc.

    // Reloading is inspired by @DanTup maneuver: https://github.com/microsoft/vscode/issues/45774#issuecomment-373423895
    ctx.registerCommand('reload', _ => async () => {
        void vscode.window.showInformationMessage('Reloading rust-analyzer...');
        await deactivate();
        while (context.subscriptions.length > 0) {
            try {
                context.subscriptions.pop()!.dispose();
            } catch (err) {
                log.error("Dispose error:", err);
            }
        }
        await activate(context).catch(log.error);
    });

    ctx.registerCommand('analyzerStatus', commands.analyzerStatus);
    ctx.registerCommand('memoryUsage', commands.memoryUsage);
    ctx.registerCommand('shuffleCrateGraph', commands.shuffleCrateGraph);
    ctx.registerCommand('reloadWorkspace', commands.reloadWorkspace);
    ctx.registerCommand('matchingBrace', commands.matchingBrace);
    ctx.registerCommand('joinLines', commands.joinLines);
    ctx.registerCommand('parentModule', commands.parentModule);
    ctx.registerCommand('syntaxTree', commands.syntaxTree);
    ctx.registerCommand('viewHir', commands.viewHir);
    ctx.registerCommand('viewItemTree', commands.viewItemTree);
    ctx.registerCommand('viewCrateGraph', commands.viewCrateGraph);
    ctx.registerCommand('viewFullCrateGraph', commands.viewFullCrateGraph);
    ctx.registerCommand('expandMacro', commands.expandMacro);
    ctx.registerCommand('run', commands.run);
    ctx.registerCommand('copyRunCommandLine', commands.copyRunCommandLine);
    ctx.registerCommand('debug', commands.debug);
    ctx.registerCommand('newDebugConfig', commands.newDebugConfig);
    ctx.registerCommand('openDocs', commands.openDocs);
    ctx.registerCommand('openCargoToml', commands.openCargoToml);
    ctx.registerCommand('peekTests', commands.peekTests);
    ctx.registerCommand('moveItemUp', commands.moveItemUp);
    ctx.registerCommand('moveItemDown', commands.moveItemDown);

    defaultOnEnter.dispose();
    ctx.registerCommand('onEnter', commands.onEnter);

    ctx.registerCommand('ssr', commands.ssr);
    ctx.registerCommand('serverVersion', commands.serverVersion);
    ctx.registerCommand('toggleInlayHints', commands.toggleInlayHints);

    // Internal commands which are invoked by the server.
    ctx.registerCommand('runSingle', commands.runSingle);
    ctx.registerCommand('debugSingle', commands.debugSingle);
    ctx.registerCommand('showReferences', commands.showReferences);
    ctx.registerCommand('applySnippetWorkspaceEdit', commands.applySnippetWorkspaceEditCommand);
    ctx.registerCommand('resolveCodeAction', commands.resolveCodeAction);
    ctx.registerCommand('applyActionGroup', commands.applyActionGroup);
    ctx.registerCommand('gotoLocation', commands.gotoLocation);
}

export async function deactivate() {
    await setContextValue(RUST_PROJECT_CONTEXT_NAME, undefined);
    await ctx?.client.stop();
    ctx = undefined;
}

async function bootstrap(context: vscode.ExtensionContext, config: Config, state: PersistentState): Promise<string> {
    const path = await getServer(context, config, state);
    if (!path) {
        throw new Error(
            "Rust Analyzer Language Server is not available. " +
            "Please, ensure its [proper installation](https://rust-analyzer.github.io/manual.html#installation)."
        );
    }

    log.info("Using server binary at", path);

    if (!isValidExecutable(path)) {
        if (config.serverPath) {
            throw new Error(`Failed to execute ${path} --version. \`config.server.path\` or \`config.serverPath\` has been set explicitly.\
            Consider removing this config or making a valid server binary available at that path.`);
        } else {
            throw new Error(`Failed to execute ${path} --version`);
        }
    }

    return path;
}

async function patchelf(dest: vscode.Uri): Promise<void> {
    await vscode.window.withProgress(
        {
            location: vscode.ProgressLocation.Notification,
            title: "Patching rust-analyzer for NixOS"
        },
        async (progress, _) => {
            const expression = `
            {srcStr, pkgs ? import <nixpkgs> {}}:
                pkgs.stdenv.mkDerivation {
                    name = "rust-analyzer";
                    src = /. + srcStr;
                    phases = [ "installPhase" "fixupPhase" ];
                    installPhase = "cp $src $out";
                    fixupPhase = ''
                    chmod 755 $out
                    patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" $out
                    '';
                }
            `;
            const origFile = vscode.Uri.file(dest.fsPath + "-orig");
            await vscode.workspace.fs.rename(dest, origFile, { overwrite: true });
            try {
                progress.report({ message: "Patching executable", increment: 20 });
                await new Promise((resolve, reject) => {
                    const handle = exec(`nix-build -E - --argstr srcStr '${origFile.fsPath}' -o '${dest.fsPath}'`,
                        (err, stdout, stderr) => {
                            if (err != null) {
                                reject(Error(stderr));
                            } else {
                                resolve(stdout);
                            }
                        });
                    handle.stdin?.write(expression);
                    handle.stdin?.end();
                });
            } finally {
                await vscode.workspace.fs.delete(origFile);
            }
        }
    );
}

async function getServer(context: vscode.ExtensionContext, config: Config, state: PersistentState): Promise<string | undefined> {
    const explicitPath = serverPath(config);
    if (explicitPath) {
        if (explicitPath.startsWith("~/")) {
            return os.homedir() + explicitPath.slice("~".length);
        }
        return explicitPath;
    };
    if (config.package.releaseTag === null) return "rust-analyzer";

    const ext = process.platform === "win32" ? ".exe" : "";
    const bundled = vscode.Uri.joinPath(context.extensionUri, "server", `rust-analyzer${ext}`);
    const bundledExists = await vscode.workspace.fs.stat(bundled).then(() => true, () => false);
    if (bundledExists) {
        let server = bundled;
        if (await isNixOs()) {
            await vscode.workspace.fs.createDirectory(config.globalStorageUri).then();
            const dest = vscode.Uri.joinPath(config.globalStorageUri, `rust-analyzer${ext}`);
            let exists = await vscode.workspace.fs.stat(dest).then(() => true, () => false);
            if (exists && config.package.version !== state.serverVersion) {
                await vscode.workspace.fs.delete(dest);
                exists = false;
            }
            if (!exists) {
                await vscode.workspace.fs.copy(bundled, dest);
                await patchelf(dest);
            }
            server = dest;
        }
        await state.updateServerVersion(config.package.version);
        return server.fsPath;
    }

    await state.updateServerVersion(undefined);
    await vscode.window.showErrorMessage(
        "Unfortunately we don't ship binaries for your platform yet. " +
        "You need to manually clone the rust-analyzer repository and " +
        "run `cargo xtask install --server` to build the language server from sources. " +
        "If you feel that your platform should be supported, please create an issue " +
        "about that [here](https://github.com/rust-analyzer/rust-analyzer/issues) and we " +
        "will consider it."
    );
    return undefined;
}

function serverPath(config: Config): string | null {
    return process.env.__RA_LSP_SERVER_DEBUG ?? config.serverPath;
}

async function isNixOs(): Promise<boolean> {
    try {
        const contents = (await vscode.workspace.fs.readFile(vscode.Uri.file("/etc/os-release"))).toString();
        return contents.indexOf("ID=nixos") !== -1;
    } catch {
        return false;
    }
}

function warnAboutExtensionConflicts() {
    if (vscode.extensions.getExtension("rust-lang.rust")) {
        vscode.window.showWarningMessage(
            `You have both the rust-analyzer (matklad.rust-analyzer) and Rust (rust-lang.rust) ` +
            "plugins enabled. These are known to conflict and cause various functions of " +
            "both plugins to not work correctly. You should disable one of them.", "Got it")
            .then(() => { }, console.error);
    }
}

/**
 * Sets up additional language configuration that's impossible to do via a
 * separate language-configuration.json file. See [1] for more information.
 *
 * [1]: https://github.com/Microsoft/vscode/issues/11514#issuecomment-244707076
 */
function configureLanguage(): vscode.Disposable {
    const indentAction = vscode.IndentAction.None;
    return vscode.languages.setLanguageConfiguration('rust', {
        onEnterRules: [
            {
                // Doc single-line comment
                // e.g. ///|
                beforeText: /^\s*\/{3}.*$/,
                action: { indentAction, appendText: '/// ' },
            },
            {
                // Parent doc single-line comment
                // e.g. //!|
                beforeText: /^\s*\/{2}\!.*$/,
                action: { indentAction, appendText: '//! ' },
            },
            {
                // Begins an auto-closed multi-line comment (standard or parent doc)
                // e.g. /** | */ or /*! | */
                beforeText: /^\s*\/\*(\*|\!)(?!\/)([^\*]|\*(?!\/))*$/,
                afterText: /^\s*\*\/$/,
                action: { indentAction: vscode.IndentAction.IndentOutdent, appendText: ' * ' },
            },
            {
                // Begins a multi-line comment (standard or parent doc)
                // e.g. /** ...| or /*! ...|
                beforeText: /^\s*\/\*(\*|\!)(?!\/)([^\*]|\*(?!\/))*$/,
                action: { indentAction, appendText: ' * ' },
            },
            {
                // Continues a multi-line comment
                // e.g.  * ...|
                beforeText: /^(\ \ )*\ \*(\ ([^\*]|\*(?!\/))*)?$/,
                action: { indentAction, appendText: '* ' },
            },
            {
                // Dedents after closing a multi-line comment
                // e.g.  */|
                beforeText: /^(\ \ )*\ \*\/\s*$/,
                action: { indentAction, removeText: 1 },
            },
        ],
    });
}
mplemented here). The ide contains a public API/façade, as well as implementation for a plethora of smaller features.

Architecture Invariant: ide crate strives to provide a perfect API. Although at the moment it has only one consumer, the LSP server, LSP does not influence its API design. Instead, we keep in mind a hypothetical ideal client -- an IDE tailored specifically for rust, every nook and cranny of which is packed with Rust-specific goodies.

crates/rust-analyzer

This crate defines the rust-analyzer binary, so it is the entry point. It implements the language server.

Architecture Invariant: rust-analyzer is the only crate that knows about LSP and JSON serialization. If you want to expose a data structure X from ide to LSP, don't make it serializable. Instead, create a serializable counterpart in rust-analyzer crate and manually convert between the two.

GlobalState is the state of the server. The main_loop defines the server event loop which accepts requests and sends responses. Requests that modify the state or might block user's typing are handled on the main thread. All other requests are processed in background.

Architecture Invariant: the server is stateless, a-la HTTP. Sometimes state needs to be preserved between requests. For example, "what is the edit for the fifth completion item of the last completion edit?". For this, the second request should include enough info to re-create the context from scratch. This generally means including all the parameters of the original request.

reload module contains the code that handles configuration and Cargo.toml changes. This is a tricky business.

Architecture Invariant: rust-analyzer should be partially available even when the build is broken. Reloading process should not prevent IDE features from working.

crates/toolchain, crates/project_model, crates/flycheck

These crates deal with invoking cargo to learn about project structure and get compiler errors for the "check on save" feature.

They use crates/path heavily instead of std::path. A single rust-analyzer process can serve many projects, so it is important that server's current directory does not leak.

crates/mbe, crates/tt, crates/proc_macro_api, crates/proc_macro_srv

These crates implement macros as token tree -> token tree transforms. They are independent from the rest of the code.

tt crate defined TokenTree, a single token or a delimited sequence of token trees. mbe crate contains tools for transforming between syntax trees and token tree. And it also handles the actual parsing and expansion of declarative macro (a-la "Macros By Example" or mbe).

For proc macros, the client-server model are used. We pass an argument --proc-macro to rust-analyzer binary to start a separate process (proc_macro_srv). And the client (proc_macro_api) provides an interface to talk to that server separately.

And then token trees are passed from client, and the server will load the corresponding dynamic library (which built by cargo). And due to the fact the api for getting result from proc macro are always unstable in rustc, we maintain our own copy (and paste) of that part of code to allow us to build the whole thing in stable rust.

Architecture Invariant: Bad proc macros may panic or segfault accidentally. So we run it in another process and recover it from fatal error. And they may be non-deterministic which conflict how salsa works, so special attention is required.

crates/cfg

This crate is responsible for parsing, evaluation and general definition of cfg attributes.

crates/vfs, crates/vfs-notify

These crates implement a virtual file system. They provide consistent snapshots of the underlying file system and insulate messy OS paths.

Architecture Invariant: vfs doesn't assume a single unified file system. i.e., a single rust-analyzer process can act as a remote server for two different machines, where the same /tmp/foo.rs path points to different files. For this reason, all path APIs generally take some existing path as a "file system witness".

crates/stdx

This crate contains various non-rust-analyzer specific utils, which could have been in std, as well as copies of unstable std items we would like to make use of already, like std::str::split_once.

crates/profile

This crate contains utilities for CPU and memory profiling.

Cross-Cutting Concerns

This sections talks about the things which are everywhere and nowhere in particular.

Stability Guarantees

One of the reasons rust-analyzer moves relatively fast is that we don't introduce new stability guarantees. Instead, as much as possible we leverage existing ones.

Examples:

Another important example is that rust-analyzer isn't run on CI, so, unlike rustc and clippy, it is actually ok for us to change runtime behavior.

At some point we might consider opening up APIs or allowing crates.io libraries to include rust-analyzer specific annotations, but that's going to be a big commitment on our side.

Exceptions:

Code generation

Some components in this repository are generated through automatic processes. Generated code is updated automatically on cargo test. Generated code is generally committed to the git repository.

In particular, we generate:

See the sourcegen crate for details.

Architecture Invariant: we avoid bootstrapping. For codegen we need to parse Rust code. Using rust-analyzer for that would work and would be fun, but it would also complicate the build process a lot. For that reason, we use syn and manual string parsing.

Cancellation

Let's say that the IDE is in the process of computing syntax highlighting, when the user types foo. What should happen? rust-analyzers answer is that the highlighting process should be cancelled -- its results are now stale, and it also blocks modification of the inputs.

The salsa database maintains a global revision counter. When applying a change, salsa bumps this counter and waits until all other threads using salsa finish. If a thread does salsa-based computation and notices that the counter is incremented, it panics with a special value (see Canceled::throw). That is, rust-analyzer requires unwinding.

ide is the boundary where the panic is caught and transformed into a Result<T, Cancelled>.

Testing

Rust Analyzer has three interesting system boundaries to concentrate tests on.

The outermost boundary is the rust-analyzer crate, which defines an LSP interface in terms of stdio. We do integration testing of this component, by feeding it with a stream of LSP requests and checking responses. These tests are known as "heavy", because they interact with Cargo and read real files from disk. For this reason, we try to avoid writing too many tests on this boundary: in a statically typed language, it's hard to make an error in the protocol itself if messages are themselves typed. Heavy tests are only run when RUN_SLOW_TESTS env var is set.

The middle, and most important, boundary is ide. Unlike rust-analyzer, which exposes API, ide uses Rust API and is intended for use by various tools. A typical test creates an AnalysisHost, calls some Analysis functions and compares the results against expectation.

The innermost and most elaborate boundary is hir. It has a much richer vocabulary of types than ide, but the basic testing setup is the same: we create a database, run some queries, assert result.

For comparisons, we use the expect crate for snapshot testing.

To test various analysis corner cases and avoid forgetting about old tests, we use so-called marks. See the marks module in the test_utils crate for more.

Architecture Invariant: rust-analyzer tests do not use libcore or libstd. All required library code must be a part of the tests. This ensures fast test execution.

Architecture Invariant: tests are data driven and do not test the API. Tests which directly call various API functions are a liability, because they make refactoring the API significantly more complicated. So most of the tests look like this:

#[track_caller]
fn check(input: &str, expect: expect_test::Expect) {
    // The single place that actually exercises a particular API
}

#[test]
fn foo() {
    check("foo", expect![["bar"]]);
}

#[test]
fn spam() {
    check("spam", expect![["eggs"]]);
}
// ...and a hundred more tests that don't care about the specific API at all.

To specify input data, we use a single string literal in a special format, which can describe a set of rust files. See the Fixture its module for fixture examples and documentation.

Architecture Invariant: all code invariants are tested by #[test] tests. There's no additional checks in CI, formatting and tidy tests are run with cargo test.

Architecture Invariant: tests do not depend on any kind of external resources, they are perfectly reproducible.

Performance Testing

TBA, take a look at the metrics xtask and #[test] fn benchmark_xxx() functions.

Error Handling

Architecture Invariant: core parts of rust-analyzer (ide/hir) don't interact with the outside world and thus can't fail. Only parts touching LSP are allowed to do IO.

Internals of rust-analyzer need to deal with broken code, but this is not an error condition. rust-analyzer is robust: various analysis compute (T, Vec<Error>) rather than Result<T, Error>.

rust-analyzer is a complex long-running process. It will always have bugs and panics. But a panic in an isolated feature should not bring down the whole process. Each LSP-request is protected by a catch_unwind. We use always and never macros instead of assert to gracefully recover from impossible conditions.

Observability

rust-analyzer is a long-running process, so it is important to understand what's going on inside. We have several instruments for that.

The event loop that runs rust-analyzer is very explicit. Rather than spawning futures or scheduling callbacks (open), the event loop accepts an enum of possible events (closed). It's easy to see all the things that trigger rust-analyzer processing, together with their performance

rust-analyzer includes a simple hierarchical profiler (hprof). It is enabled with RA_PROFILE='*>50' env var (log all (*) actions which take more than 50 ms) and produces output like:

85ms - handle_completion
    68ms - import_on_the_fly
        67ms - import_assets::search_for_relative_paths
             0ms - crate_def_map:wait (804 calls)
             0ms - find_path (16 calls)
             2ms - find_similar_imports (1 calls)
             0ms - generic_params_query (334 calls)
            59ms - trait_solve_query (186 calls)
         0ms - Semantics::analyze_impl (1 calls)
         1ms - render_resolution (8 calls)
     0ms - Semantics::analyze_impl (5 calls)

This is cheap enough to enable in production.

Similarly, we save live object counting (RA_COUNT=1). It is not cheap enough to enable in prod, and this is a bug which should be fixed.

Configurability

rust-analyzer strives to be as configurable as possible while offering reasonable defaults where no configuration exists yet. There will always be features that some people find more annoying than helpful, so giving the users the ability to tweak or disable these is a big part of offering a good user experience. Mind the code--architecture gap: at the moment, we are using fewer feature flags than we really should.

Serialization

In Rust, it is easy (often too easy) to add serialization to any type by adding #[derive(Serialize)]. This easiness is misleading -- serializable types impose significant backwards compatability constraints. If a type is serializable, then it is a part of some IPC boundary. You often don't control the other side of this boundary, so changing serializable types is hard.

For this reason, the types in ide, base_db and below are not serializable by design. If such types need to cross an IPC boundary, then the client of rust-analyzer needs to provide custom, client-specific serialization format. This isolates backwards compatibility and migration concerns to a specific client.

For example, rust-project.json is it's own format -- it doesn't include CrateGraph as is. Instead, it creates a CrateGraph by calling appropriate constructing functions.