Console #189 - Interview with Elia of Opal - a Ruby to JavaScript source-to-source compiler

Featuring qsv, OSMnx, and more

🤝 Sponsor

This space is reserved for sponsors that support us to keep the newsletter going! Want to support Console? Send us a note at osh@codesee.io

---

🏗️ Projects

Browse through open source projects on OpenSourceHub.io, add your project to get more exposure and connect with other maintainers and contributors!

qsv

qsv (Quicksilver) is a command line program for querying, indexing, slicing, analyzing, filtering, enriching, transforming, sorting, validating & joining CSV files.

language: Rust stars: 1085 last commit: today
repo: github.com/jqnatividad/qsv
site: qsv.dathere.com

OSMnx

OSMnx is a Python package to easily download, model, analyze, and visualize street networks and other geospatial features from OpenStreetMap.

language: Python stars: 4478 last commit: 2 weeks
repo: github.com/gboeing/osmnx
site: osmnx.readthedocs.io

Opal

Opal is a Ruby to JavaScript source-to-source compiler. It also has an implementation of the Ruby corelib and stdlib

language: Ruby stars: 4780 last commit: a month
repo: github.com/opal/opal
site: opalrb.com

---

Join thousands of other open-source enthusiasts and developers in the Open Source Hub Discord server to continue the discussion on the projects in this week's email!

---

🎙️ Interview with Elia of Opal - a Ruby to JavaScript source-to-source compiler

Hi Elia, Let’s start with your background! Where are you from, where have you worked in the past, how did you learn to program, and what languages or frameworks do you like?

Hi! I’m from Italy, I’ve been working as a software developer for about 15 years now, and I'm not one of those early childhood programmers that started with Basic and a Commodore 64, I had my initial encounter with programming in high school, whe I was really fascinated by DHTML and the ability to make things move on the screen, at the time I printed and read the online Netscape guide to JavaScript with really poor understanding of technical English and without understanding much of it.

What’s your most controversial programming opinion?

I don't know if it's controversial, but I was always attracted to making things beautiful for the user, or code reader, and that's what got me hooked into Ruby, it's really a beautiful language, with the same beauty of some mathematical formulas, and I think that's what makes it so appealing to many people. From this kind of love for the next person experiencing your app or code I think all sorts of good things come out, like the attention to details, the care for the user experience, the care for the code readability, but also the right amount of pragmatism to make things work and be useful. Having the individual person at the center is what matters, not a group or a "higher good" or a "greater good", but the single person, you can always sacrifice an individual person for the "greater good" but when you care for the next person you can't do that. I like that Ruby is released yearly on Christmas day, that's ultimately the message of Christmas, and the drive behind Ruby.

What is your favorite software tool?

I think I owe a lot to TextMate, and its main merits are two. The first one is that it's so well integrated with macOS that just by learning it you learn a lot about the OS, ultimately making it your IDE, fully espousing the consistency and "just works" approach that made Apple so successful. The second one is its extremely elegant way of solving extendability, going full on with the Unix philosophy of small commands that can interact with each other through piping and environment variables, I am still amazed that to this day the commenting in TextMate is handled by a Ruby script. The whole programming community owes it a lot, and modern VSCode is still full of references to TextMate throughout its documentation, as a testament to the profound impact it had.

What are you currently learning?

Programming wise and outside of my main open-source work I'm having some fun reimplementing TextMate features on VSCode, those are nice little puzzles to solve that have a practical application. I'm likely the only user to my extension for TextMate expats but it's ok, I kinda like being in such a niche.

Why was Opal started?

Can't say for sure as I wasn't the original author. But I can say how I bumped into it. It was circa 2011 and I was working on my first JavaScript-only project. At the time we still had to support IE7 and Coffescript was the new kid on the block, so new that we didn't dare use it on client's code. After falling for most of the now infamous JavaScript "WAT" moments I was looking for something better, and started researching projects compiling Ruby to JavaScript. I found a few, but Opal was the only one that was actually usable, and I started polishing it so I could use it in my projects. In 2012 I had the first bit of Ruby code running in production on the browser, and writing Ruby code for the browser was a real joy. I was hooked, and I started contributing more and more to Opal, eventually becoming the maintainer in 2015.

How does Opal work?

The way Opal works is very similar to most other languages compiling to JavaScript. It all starts with parsing code into what's called an abstract syntax tree, which then gets transformed into JavaScript code. The one thing that sets Opal apart from other attempts at porting Ruby to the browser is that it tries to fully leverage the underlying JavaScript engine by reusing its native objects and methods. This means that when you write a = [1, 2, 3] in Ruby, Opal will actually create a JavaScript array, and when you call a.map { |x| x * 2 } Opal will actually call the native JavaScript map method on the array passing an anonymous function that multiplies the value by 2. This approach has a few advantages, the first one is that it's really fast, the second one is that it's really small, and the third one is that it's really easy to debug. Just recently we were able to reimplement the Ruby Hash class using the native Map object, and this resulted in both speed and size improvements.

Where did the name for Opal come from?

The name was chosen in reference to the Ruby gemstone by the original author, Adam Beynon, referencing the famous gemstone that can be found in Australia. Of course the reference is I think it's a great name, Opal gemstones are really beautiful and I love Australia.

Who, or what was the biggest inspiration for Opal?

For me Coffeescript was really instrumental in opening my eyes to the possibility of writing Ruby code for the browser, I think Adam had a few other projects compiling Objective-C and other languages into JavaScript.

Are there any overarching goals of Opal that drive design or implementation? If so, what trade-offs have been made in Opal as a consequence of these goals?

This kind of venture is deemed to have lots of trade-offs, for example in early days one great Ruby feature like method_missing wasn't supported. Surprisingly though, we've seen contributions for many things that previously seemed impossible. Although we have great support for Node.js and other server-side environments, our main target platform is still the browser.

What is the most challenging problem that’s been solved in Opal, so far?

I think the most challenging problem was module inclusion (either via include or prepend) and the super keyword. The reason is that Ruby has a very complex inheritance model, and it's not easy to mirror that in JavaScript. We've been able to do it tho, using the prototype chain. That part is very challenging to reason about given its meta nature, but we need to give credit to JavaScript for being so flexible that a solution was eventually found.

For anyone interested in the details, I strongly encourage giving a read to the official Hacking guide and to the parts of the runtime.js file dedicated to core object and classes initialization.

Are there any competitors or projects similar to Opal? If so, what were they lacking that made you consider building something new?

If we're willing to define it so, we've recently got a newer "competitor" in the official Ruby support for WebAssembly, but I wouldn't treat it as a real competitor as our mission is to get Ruby on the browser and we can only root for an official viable solution. On the other hand it's still in its early days and, given its size, it's not suitable for generic web-application code except for a few good use cases.

What was the most surprising thing you learned while working on Opal?

I think the most surprising thing is how much you can do with so little code. The core of Opal is really small, and it's really surprising how much you can do with it. I think it's a testament to the power of Ruby, and the power of JavaScript as well. I also learned so much about both Ruby and JavaScript, that I think trying to implement additional features in Opal is a great way to improve both skills for any Ruby developer also working with JavaScript.

What is your typical approach to debugging issues filed in the Opal repo?

We have a weekly meeting where we discuss issues and PRs, and we try to give feedback as soon as possible. We do our best to stay abreast of everything but sometimes we miss things, and we're always happy to be pinged on issues and PRs. We also have a Slack channel where we're always happy to help.

What is the release process like for Opal?

We keep track of unreleased changelog entry in a separate file and we have a few scripts that will merge that into the main changelog file and bump the version number. We then tag the release and push it to Ruby gems using Bundler's gem helper tasks. We also have a few other scripts that will update the website and the documentation. The whole process is pretty smooth and we're able to release a new version in a matter of minutes.

Is Opal intended to eventually be monetized if it isn’t monetized already? If so, how? If it’s already monetized, what is your main source of revenue?

Opal is not monetized, and it's not intended to be, at least directly. Being a language implementation it won't have any monetizable application on its own. We maintain a funding page that we use to sustain infrastructure costs and possibly a website redesign, but that's about it. We're always open to sponsorships and donations and I think that has been the case for some fixes and recent features, I think that's a great model for open-source projects of this kind. I'm also personally thankful to my employer, Nebulab, for allowing me to spend some of my time on Opal.

What are you most proud of?

At some point I had a few ecommerce stores fully running their JavaScript code on Opal with a homegrown framework written in a hundred lines of Ruby. Those stores were very large fashion ecommerce stores serving tens of thousands of users a day.

How do you balance your work on open-source with your day job and other responsibilities?

I realized early on that I need to be in for the marathon and not the sprint, so I keep it best-effort and I try to keep it fun, having maintainers burn out it's a lose-lose scenario for any open-source project. Open source is a gift to the world, by the maintainers, I'm grateful for all the open-source I have the privilege to use, and I think this is a healthy attitude toward it.

What is the best way for a new developer to contribute to Opal?

I think the best way is to start using it, and report any issue or missing feature. If anyone wants to go deeper it's a treasure trove of puzzles to solve, thanks to Ruby's spec project providing a comprehensive test suite for the language. As mentioned above also have a Hacking guide that can help anyone get started.

If you plan to continue developing Opal, where do you see the project heading next?

There are some themes that we'd like to explore and some features we need to support better. We're looking into things like dead-code-elimination for reducing the bundle size even more for compiled projects and into improving the support for modules and importmaps. We're also exploring the possibility of supporting the new Ruby WebAssembly implementation featuring a compatible API so it's easier to write code that can run both. But there are so many ideas that it is hard to come up with a full list, things like using JavaScript symbols for Ruby symbols and method ids, automatic JavaScript libraries wrappers, and so on.

What motivates you to continue contributing to Opal?

After all these years I still love a good puzzle and I still love Ruby, it keeps me sharp on the language and on JavaScript, and although I regret not having any big production app running on Opal anymore, I still cherish providing such a great tool to the people using it everyday.

Are there any other projects besides Opal that you’re working on?

As part of my day Job I work every day on Solidus, the Rails e-commerce framework, lately I also had some involvement in View Component, and I wrote an ERB-formatter for VSCode to scratch my own itch that got some traction. As mentioned, I'm also maintaining a VSCode extension for stranded TextMate users that I use myself, but I jokingly gave it 5 stars on the VSCode marketplace because I really enjoy it .

Where do you see software development heading next?

Everyone's talking about AI, and I won't make an exception. As many I swing between interpreting my fears of it taking over the world in a Terminator-like fashion and the hope that it's a sort of luddite instinct that will eventually turn into developers just being more productive.

Where do you see open-source heading next?

I think open-source greatly increased efficiency and thus value-generation across the world, and at the same time it opened opportunities for many people that otherwise wouldn't have had them. I think it's a great thing and I hope it will continue to exist and thrive.

---

Want to join the conversation about one of the projects featured this week? Drop a comment, or see what others are saying!

Interested in sponsoring the newsletter or know of any cool projects or interesting developers you want us to interview? Reach out at osh@codesee.io or mention us @ConsoleWeekly!