Unnamed repository; edit this file 'description' to name the repository.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
//! File and span related types.
use std::fmt::{self, Write};

mod ast_id;
mod hygiene;
mod map;

pub use self::{
    ast_id::{AstIdMap, AstIdNode, ErasedFileAstId, FileAstId},
    hygiene::{SyntaxContext, Transparency},
    map::{RealSpanMap, SpanMap},
};

pub use syntax::Edition;
pub use text_size::{TextRange, TextSize};
pub use vfs::FileId;

// The first index is always the root node's AstId
/// The root ast id always points to the encompassing file, using this in spans is discouraged as
/// any range relative to it will be effectively absolute, ruining the entire point of anchored
/// relative text ranges.
pub const ROOT_ERASED_FILE_AST_ID: ErasedFileAstId = ErasedFileAstId::from_raw(0);

/// FileId used as the span for syntax node fixups. Any Span containing this file id is to be
/// considered fake.
pub const FIXUP_ERASED_FILE_AST_ID_MARKER: ErasedFileAstId =
    // we pick the second to last for this in case we ever consider making this a NonMaxU32, this
    // is required to be stable for the proc-macro-server
    ErasedFileAstId::from_raw(!0 - 1);

pub type Span = SpanData<SyntaxContext>;

impl Span {
    pub fn cover(self, other: Span) -> Span {
        if self.anchor != other.anchor {
            return self;
        }
        let range = self.range.cover(other.range);
        Span { range, ..self }
    }
}

/// Spans represent a region of code, used by the IDE to be able link macro inputs and outputs
/// together. Positions in spans are relative to some [`SpanAnchor`] to make them more incremental
/// friendly.
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
pub struct SpanData<Ctx> {
    /// The text range of this span, relative to the anchor.
    /// We need the anchor for incrementality, as storing absolute ranges will require
    /// recomputation on every change in a file at all times.
    pub range: TextRange,
    /// The anchor this span is relative to.
    pub anchor: SpanAnchor,
    /// The syntax context of the span.
    pub ctx: Ctx,
}

impl<Ctx: fmt::Debug> fmt::Debug for SpanData<Ctx> {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        if f.alternate() {
            fmt::Debug::fmt(&self.anchor.file_id.file_id().index(), f)?;
            f.write_char(':')?;
            fmt::Debug::fmt(&self.anchor.ast_id.into_raw(), f)?;
            f.write_char('@')?;
            fmt::Debug::fmt(&self.range, f)?;
            f.write_char('#')?;
            self.ctx.fmt(f)
        } else {
            f.debug_struct("SpanData")
                .field("range", &self.range)
                .field("anchor", &self.anchor)
                .field("ctx", &self.ctx)
                .finish()
        }
    }
}

impl<Ctx: Copy> SpanData<Ctx> {
    pub fn eq_ignoring_ctx(self, other: Self) -> bool {
        self.anchor == other.anchor && self.range == other.range
    }
}

impl fmt::Display for Span {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        fmt::Debug::fmt(&self.anchor.file_id.file_id().index(), f)?;
        f.write_char(':')?;
        fmt::Debug::fmt(&self.anchor.ast_id.into_raw(), f)?;
        f.write_char('@')?;
        fmt::Debug::fmt(&self.range, f)?;
        f.write_char('#')?;
        self.ctx.fmt(f)
    }
}

#[derive(Copy, Clone, PartialEq, Eq, Hash)]
pub struct SpanAnchor {
    pub file_id: EditionedFileId,
    pub ast_id: ErasedFileAstId,
}

impl fmt::Debug for SpanAnchor {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.debug_tuple("SpanAnchor").field(&self.file_id).field(&self.ast_id.into_raw()).finish()
    }
}

/// A [`FileId`] and [`Edition`] bundled up together.
/// The MSB is reserved for `HirFileId` encoding, more upper bits are used to then encode the edition.
#[derive(Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct EditionedFileId(u32);

impl fmt::Debug for EditionedFileId {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.debug_tuple("EditionedFileId").field(&self.file_id()).field(&self.edition()).finish()
    }
}

impl From<EditionedFileId> for FileId {
    fn from(value: EditionedFileId) -> Self {
        value.file_id()
    }
}

const _: () = assert!(
    EditionedFileId::RESERVED_HIGH_BITS
        + EditionedFileId::EDITION_BITS
        + EditionedFileId::FILE_ID_BITS
        == u32::BITS
);
const _: () = assert!(
    EditionedFileId::RESERVED_MASK ^ EditionedFileId::EDITION_MASK ^ EditionedFileId::FILE_ID_MASK
        == 0xFFFF_FFFF
);

impl EditionedFileId {
    pub const RESERVED_MASK: u32 = 0x8000_0000;
    pub const EDITION_MASK: u32 = 0x7F80_0000;
    pub const FILE_ID_MASK: u32 = 0x007F_FFFF;

    pub const MAX_FILE_ID: u32 = Self::FILE_ID_MASK;

    pub const RESERVED_HIGH_BITS: u32 = Self::RESERVED_MASK.count_ones();
    pub const FILE_ID_BITS: u32 = Self::FILE_ID_MASK.count_ones();
    pub const EDITION_BITS: u32 = Self::EDITION_MASK.count_ones();

    pub const fn current_edition(file_id: FileId) -> Self {
        Self::new(file_id, Edition::CURRENT)
    }

    pub const fn new(file_id: FileId, edition: Edition) -> Self {
        let file_id = file_id.index();
        let edition = edition as u32;
        assert!(file_id <= Self::MAX_FILE_ID);
        Self(file_id | (edition << Self::FILE_ID_BITS))
    }

    pub fn from_raw(u32: u32) -> Self {
        assert!(u32 & Self::RESERVED_MASK == 0);
        assert!((u32 & Self::EDITION_MASK) >> Self::FILE_ID_BITS <= Edition::LATEST as u32);
        Self(u32)
    }

    pub const fn as_u32(self) -> u32 {
        self.0
    }

    pub const fn file_id(self) -> FileId {
        FileId::from_raw(self.0 & Self::FILE_ID_MASK)
    }

    pub const fn unpack(self) -> (FileId, Edition) {
        (self.file_id(), self.edition())
    }

    pub const fn edition(self) -> Edition {
        let edition = (self.0 & Self::EDITION_MASK) >> Self::FILE_ID_BITS;
        debug_assert!(edition <= Edition::LATEST as u32);
        unsafe { std::mem::transmute(edition as u8) }
    }
}

#[cfg(not(feature = "salsa"))]
mod salsa {
    #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
    pub struct Id(u32);
}

/// Input to the analyzer is a set of files, where each file is identified by
/// `FileId` and contains source code. However, another source of source code in
/// Rust are macros: each macro can be thought of as producing a "temporary
/// file". To assign an id to such a file, we use the id of the macro call that
/// produced the file. So, a `HirFileId` is either a `FileId` (source code
/// written by user), or a `MacroCallId` (source code produced by macro).
///
/// What is a `MacroCallId`? Simplifying, it's a `HirFileId` of a file
/// containing the call plus the offset of the macro call in the file. Note that
/// this is a recursive definition! However, the size_of of `HirFileId` is
/// finite (because everything bottoms out at the real `FileId`) and small
/// (`MacroCallId` uses the location interning. You can check details here:
/// <https://en.wikipedia.org/wiki/String_interning>).
///
/// Internally this holds a `salsa::Id`, but we cannot use this definition here
/// as it references things from base-db and hir-expand.
// FIXME: Give this a better fitting name
#[derive(Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct HirFileId(pub salsa::Id);

/// `MacroCallId` identifies a particular macro invocation, like
/// `println!("Hello, {}", world)`.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct MacroCallId(pub salsa::Id);

/// Legacy span type, only defined here as it is still used by the proc-macro server.
/// While rust-analyzer doesn't use this anymore at all, RustRover relies on the legacy type for
/// proc-macro expansion.
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
pub struct TokenId(pub u32);

impl std::fmt::Debug for TokenId {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        self.0.fmt(f)
    }
}
rr, but the stderr itself is processed by VS Code. `--log-file <PATH>` CLI argument allows logging to file. Setting the `RA_LOG_FILE=<PATH>` environment variable will also log to file, it will also override `--log-file`. To see stderr in the running VS Code instance, go to the "Output" tab of the panel and select `rust-analyzer`. This shows `eprintln!` as well. Note that `stdout` is used for the actual protocol, so `println!` will break things. To log all communication between the server and the client, there are two choices: * You can log on the server side, by running something like ``` env RA_LOG=lsp_server=debug code . ``` * You can log on the client side, by the `rust-analyzer: Toggle LSP Logs` command or enabling `"rust-analyzer.trace.server": "verbose"` workspace setting. These logs are shown in a separate tab in the output and could be used with LSP inspector. Kudos to [@DJMcNab](https://github.com/DJMcNab) for setting this awesome infra up! There are also several VS Code commands which might be of interest: * `rust-analyzer: Status` shows some memory-usage statistics. * `rust-analyzer: Syntax Tree` shows syntax tree of the current file/selection. * `rust-analyzer: View Hir` shows the HIR expressions within the function containing the cursor. You can hover over syntax nodes in the opened text file to see the appropriate rust code that it refers to and the rust editor will also highlight the proper text range. If you trigger Go to Definition in the inspected Rust source file, the syntax tree read-only editor should scroll to and select the appropriate syntax node token. ![demo](https://user-images.githubusercontent.com/36276403/78225773-6636a480-74d3-11ea-9d9f-1c9d42da03b0.png) ## Profiling We have a built-in hierarchical profiler, you can enable it by using `RA_PROFILE` env-var: ``` RA_PROFILE=* // dump everything RA_PROFILE=foo|bar|baz // enabled only selected entries RA_PROFILE=*@3>10 // dump everything, up to depth 3, if it takes more than 10 ms ``` Some rust-analyzer contributors have `export RA_PROFILE='*>10'` in my shell profile. For machine-readable JSON output, we have the `RA_PROFILE_JSON` env variable. We support filtering only by span name: ``` RA_PROFILE=* // dump everything RA_PROFILE_JSON="vfs_load|parallel_prime_caches|discover_command" // dump selected spans ``` We also have a "counting" profiler which counts number of instances of popular structs. It is enabled by `RA_COUNT=1`. To measure time for from-scratch analysis, use something like this: ``` $ cargo run --release -p rust-analyzer -- analysis-stats ../chalk/ ``` For measuring time of incremental analysis, use either of these: ``` $ cargo run --release -p rust-analyzer -- analysis-bench ../chalk/ --highlight ../chalk/chalk-engine/src/logic.rs $ cargo run --release -p rust-analyzer -- analysis-bench ../chalk/ --complete ../chalk/chalk-engine/src/logic.rs:94:0 ``` Look for `fn benchmark_xxx` tests for a quick way to reproduce performance problems. ## Release Process Release process is handled by `release`, `dist`, `publish-release-notes` and `promote` xtasks, `release` being the main one. `release` assumes that you have checkouts of `rust-analyzer`, `rust-analyzer.github.io`, and `rust-lang/rust` in the same directory: ``` ./rust-analyzer ./rust-analyzer.github.io ./rust-rust-analyzer # Note the name! ``` The remote for `rust-analyzer` must be called `upstream` (I use `origin` to point to my fork). In addition, for `xtask promote` (see below), `rust-rust-analyzer` must have a `rust-analyzer` remote pointing to this repository on GitHub. `release` calls the GitHub API calls to scrape pull request comments and categorize them in the changelog. This step uses the `curl` and `jq` applications, which need to be available in `PATH`. Finally, you need to obtain a GitHub personal access token and set the `GITHUB_TOKEN` environment variable. Release steps: 1. Set the `GITHUB_TOKEN` environment variable. 2. Inside rust-analyzer, run `cargo xtask release`. This will: * checkout the `release` branch * reset it to `upstream/nightly` * push it to `upstream`. This triggers GitHub Actions which: * runs `cargo xtask dist` to package binaries and VS Code extension * makes a GitHub release * publishes the VS Code extension to the marketplace * call the GitHub API for PR details * create a new changelog in `rust-analyzer.github.io` 3. While the release is in progress, fill in the changelog. 4. Commit & push the changelog. 5. Run `cargo xtask publish-release-notes <CHANGELOG>` -- this will convert the changelog entry in AsciiDoc to Markdown and update the body of GitHub Releases entry. 6. Tweet. 7. Make a new branch and run `cargo xtask rustc-pull`, open a PR, and merge it. This will pull any changes from `rust-lang/rust` into `rust-analyzer`. 8. Switch to `master`, pull, then run `cargo xtask rustc-push --rust-path ../rust-rust-analyzer --rust-fork matklad/rust`. Replace `matklad/rust` with your own fork of `rust-lang/rust`. You can use the token to authenticate when you get prompted for a password, since `josh` will push over HTTPS, not SSH. This will push the `rust-analyzer` changes to your fork. You can then open a PR against `rust-lang/rust`. Note: besides the `rust-rust-analyzer` clone, the Josh cache (stored under `~/.cache/rust-analyzer-josh`) will contain a bare clone of `rust-lang/rust`. This currently takes about 3.5 GB. This [HackMD](https://hackmd.io/7pOuxnkdQDaL1Y1FQr65xg) has details about how `josh` syncs work. If the GitHub Actions release fails because of a transient problem like a timeout, you can re-run the job from the Actions console. If it fails because of something that needs to be fixed, remove the release tag (if needed), fix the problem, then start over. Make sure to remove the new changelog post created when running `cargo xtask release` a second time. We release "nightly" every night automatically and promote the latest nightly to "stable" manually, every week. We don't do "patch" releases, unless something truly egregious comes up. To do a patch release, cherry-pick the fix on top of the current `release` branch and push the branch. There's no need to write a changelog for a patch release, it's OK to include the notes about the fix into the next weekly one. Note: we tag releases by dates, releasing a patch release on the same day should work (by overwriting a tag), but I am not 100% sure. ## Permissions There are three sets of people with extra permissions: * rust-analyzer GitHub organization [**admins**](https://github.com/orgs/rust-analyzer/people?query=role:owner) (which include current t-compiler leads). Admins have full access to the org. * [**review**](https://github.com/orgs/rust-analyzer/teams/review) team in the organization. Reviewers have `r+` access to all of organization's repositories and publish rights on crates.io. They also have direct commit access, but all changes should via bors queue. It's ok to self-approve if you think you know what you are doing! bors should automatically sync the permissions. Feel free to request a review or assign any PR to a reviewer with the relevant expertise to bring the work to their attention. Don't feel pressured to review assigned PRs though. If you don't feel like reviewing for whatever reason, someone else will pick the review up! * [**triage**](https://github.com/orgs/rust-analyzer/teams/triage) team in the organization. This team can label and close issues. Note that at the time being you need to be a member of the org yourself to view the links.