Preamble
Developing high-performance systems is a mind-twisting task. We face endless clever or stupid cases to conquer. It could be meaningful that we leave thoughts here to benefit all.
This series hope to present that fast-reading tips, from a viewpoint of real engineering, to reflect the problem we meet in this project and how we solve them in an elegant way by the modern Rust.
The idea of this series is inspired by our recent Rust Chinese community’s conferences and online talkings. I really love our Rust community and thanks for all the help got from the community.
Proc Macro
Proc macro is an relatively unique language characteristics in Rust. It is great for library writers but notorious to figure out the problem when bugs happen. Unfortunately, bugs always happen… So, it should be careful to use self-brewed proc macros in large engineering.
Sometimes, the desires to have more advanced representations exceed the fear from coding for the proc macro. You start to try!
Commonly you’ve known, cargo expand and eprintln![1]. They are generally good with a little boring works to make them inconvenient. Occasionally, they meet problems for temporary broken IDEs or that you just do not want invasive prints.
Here, I suggest two other ways which I used in the Base engineering.
Proc_macro_diagnostic API
The nightly has introduced “Procedural Macro Diagnostics” APIs [3] under the feature “proc_macro_diagnostic” as friendly diag-info-show tool which is seamlessly integrated into the proc macro output.
Here, I use the s! macro in TensorBase as an example, which makes your writing C, Java like codes in your Rust sources in a free style. (In fact, this is just a raw token container with in-Rust value interpolation, you can embed almost any language in Rust using it.)
A normal working scenario like this:
Expected gen-ed code is this:
When I miss a delimiter $ for dsadsa, then it just panics,
Here, we may not quickly understand the exact problem in that we have no useful indication.
Then, we use diag APIs in the potential key parsing points, like:
And try another case - interpolated variable typo, we got this:
We find that is an error prompt immediately in vscode/RA, which is shown that what’s the problem identity and the location/span of this ident (Note: here the span is not exact which may be a bug or just a surprise).
By changing the API call from “error” to “warning”, we got a “non-blocking” warning style prompt like this:
There are four APIs: error, warning, help and note on Span for your favor. Consult the tracking issue for more[3] .
In a real engineering, it is not hard to provide an in-Rust direct external language editing experience to use that language compiler(e.g. GCC, JavaC) on top of this diagnostics API.
Great user-friendly proc macro diag experience!
Unit Testability for Proc Macro
Generally, the proc macro is just one compiler plugin kind to run at compilation time. However, it is even hard to figure out a good entrance for this kind debugging in that we are not language developers.
Another not-well-known way is, our core team gradually makes the proc macro unit testable (WIP).
You just write a unit-testability-friendly tests like your conventional unit tests, for example, unit tests for above s! macro test.
The advantage of unit test is that you use every swiss knifes in your toolbox with a right engineering style(no any adhoc setup).
For example, I enable the live debugging to the test to see what I have in kinds of proc macro syntax objects, like TokenStream here just by one click in RA:
Neat! Do not need endless prints in you code any more!
Finally, it is hopedsh fearless proc macro programming coming soon:)