Microsoft's Roslyn: Reinventing the compiler as we know it

New, experimental compilers for .Net add dynamic compilation and interactive development capabilities to C# and Visual Basic

Whatever you may think of its business practices, Microsoft has always been top-notch when it comes to developer tools. Visual Studio is widely hailed as the best IDE out there, and .Net is an intelligently designed platform that borrows the best of what Java has to offer and takes it a few steps further.

No doubt that's why so many developers got nervous when Microsoft started touting JavaScript and HTML5 as application development tools earlier this year. Those fears were compounded at this year's Build conference (née PDC), when all the buzz seemed to be about Metro-style apps rather than traditional desktop software. Some developers even worried that the new emphasis on Web technologies and the new WinRT APIs meant Microsoft was forsaking .Net altogether.

[ Go deep into HTML5 programming in InfoWorld's "HTML5 Megaguide Deep Dive" PDF how-to report. Then understand the issues surrounding HTML5 today in InfoWorld's HTML5 Deep Dive PDF strategy report. | Get software development news and insights from InfoWorld's Developer World newsletter. ]

Nothing could be further from the truth. Looking past the Metro hype, the Build conference also revealed promising road maps for C#, Visual Studio, and the .Net platform as a whole.

Perhaps the most exciting demo of the conference for .Net developers, however, was Project Roslyn, a new technology that Microsoft made available yesterday as a Community Technology Preview (CTP). Roslyn aims to bring powerful new features to C#, Visual Basic, and Visual Studio, but it's really much more than that. If it succeeds, it will reinvent how we view compilers and compiled languages altogether.

Deconstructing the compiler
Roslyn has been described as "compiler-as-a-service technology," a term that's caused a lot of confusion. I've even seen headlines heralding the project as "Microsoft's cloud compiler service" or "bringing .Net to the cloud." None of that is correct. Technically, it would be possible to offer code compilation as a cloud-based service, but it's hard to see the advantage, except in special circumstances.

Roslyn isn't services in the sense of software-as-a-service (SaaS), platform-as-a-service (PaaS), or similar cloud offerings. Rather, it's services in the sense of Windows services. Roslyn is a complete reengineering of Microsoft's .Net compiler toolchain in a new way, such that each phase of the code compilation process is exposed as a service that can be consumed by other applications.

As Microsoft's Anders Hejlsberg explained in a Build conference session, "Traditionally, a compiler is just sort of a black box. On one side you feed it source files, magic happens, and out the other end comes object files, or assemblies, or whatever the output format is."

Internally, however, there's a lot more going on. Typically, first the compiler parses your source code and breaks it down into a syntax tree. Then it builds a list of all the symbols in your program. Then it begins binding the symbols with the appropriate objects and so on.

An ordinary compiler discards all of this intermediate information once the final code is output. But with Roslyn-enabled compilers, the data from each step is accessible via its own .Net APIs. For example, a call to one API will return the entire syntax tree of a given piece of code as an object. A call to another API might return the number of methods in the code.

So what is Roslyn good for?
The most obvious advantage of this kind of "deconstructed" compiler is that it allows the entire compile-execute process to be invoked from within .Net applications. Hejlsberg demonstrated a C# program that passed a few code snippets to the C# compiler as strings; the compiler returned the resulting IL assembly code as an object, which was then passed to the Common Language Runtime (CLR) for execution. Voilà! With Roslyn, C# gains a dynamic language's ability to generate and invoke code at runtime.

Put that same code into a loop that accepts input from the user, and you've created a fully interactive read-eval-print loop (REPL) console for C#, allowing you to manipulate and experiment with .Net APIs and objects in real time. With the Roslyn technology, C# may still be a compiled language, but it effectively gains all the flexibility and expressiveness that dynamic languages such as Python and Ruby have to offer.

1 2 Page 1
From CIO: 8 Free Online Courses to Grow Your Tech Skills
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.