Console
Internet in a Box, Nushell, and Handsfree
Sponsorship
Baronfig
Did you know that you generate 4x more ideas with pen to paper than typing? Use Baronfig's all-in-one setup, the Idea Toolset, and get your ideas flowing. Visit baronfig.com »
Not subscribed to Console? Subscribe now to get a list of new open-source projects curated by an Amazon engineer in your email every week.
Already subscribed? Refer 10 friends to Console and we’ll donate $100 to an open-source project of your choice!
Projects
nushell
nushell is…you guessed it…a new type of shell.
language: Rust, stars: 14398, watchers: 147, forks: 648, issues: 504
last commit: July 01, 2021, first commit: May 10, 2019
https://twitter.com/jntrnr & https://twitter.com/andras_io
handsfree
Handsfree.js allows you to quickly integrate face, hand, and/or pose tracking to your frontend projects.
language: JavaScript, stars: 1977, watchers: 26, forks: 76, issues: 33
last commit: March 13, 2021, first commit: November 05, 2019
https://twitter.com/goinghandsfree
iiab
internet-in-a-box is a “learning hotspot” that brings the Internet’s crown jewels (Wikipedia in any language, thousands of Khan Academy videos, zoomable OpenStreetMap, electronic books, WordPress journaling, Toys from Trash electronics projects, ETC) to those without Internet. internet-in-a-box can be hosted on a Raspberry Pi.
language: Jinja, stars: 427, watchers: 14, forks: 47, issues: 226
last commit: July 01, 2021, first commit: May 27, 2017
An Interview With Oz Ramos of Handsfree
Hey Oz! Thanks for joining us! Let’s start with your background. How did you first learn how to program?
I started learning to code in the summer between grades 5 and 6. At the time my father was working on a Bulletin Board System that specialized in ASCII based games (as I remember it). I asked him to show me how to make games and he gave me a book titled “BASIC: For Dummies”.
As he often called me a dummy, I took this as an insult as if he was calling me a “basic dummy” and I tore into that book! By the end of the summer I had made a pixel art app: I didn’t know how to capture arrow keys in BASIC so I used the - and + keys to select pixels and then a number to put a colored pixel.
Who or what are your biggest influences as a developer?
My heroes growing up were Bruce Lee, Rodney Mullen, and Isaac Newton and from them I’ve adopted a kind of “endurance” for working hard on projects for long stretches with little or no personal gain.
Later I was inspired by the CodingHorror blog and the explosive rise of jQuery. Johnny Lee’s wiimote demo was also massively influential. Recently, Golan Levin showed me how to take a more creative approach with code.
If you had to suggest 1 person developers should follow, who would it be?
Definitely and without hesitation: Charlie Gerard @devdevcharlie
How do you separate good project ideas from bad ones?
Excluding projects that hurt someone or something, I don’t think there are bad project ideas...just ideas that fulfill your current goals more than others.
I organize my ideas on Notion into different tiers: long, short, and silly. The long term ideas help me steer Handsfree.js, and my short term ideas help me execute. The “fun” ideas are there for when I need a break or want to get creative.
Why was Handsfree started?
Handsfree.js was started in 2018 during my 2nd year of homelessness after I was inspired by a resident at the shelter who suffered a severe stroke. Originally the idea was to help him control a mouse pointer with his face so that he could use the web to do things like go on Facebook and watch YouTube.
However, as I used the tool myself to explore different sites I became inspired by the idea of expressively browsing the web...that is to say, to browse the web in the way you felt.
The project originated as SeeClarke.js after Arthur C Clarke who said: “Any sufficiently advanced technology is indistinguishable from magic”. The goal was to create a library that was magical to use - from reading the docs to actually building with it - for the purpose of creating assistive technologies.
But the more I used the library myself the more fun I had and I began exploring other modalities like hand tracking and even fully body pose tracking. Eventually, I renamed the project Handsfree.js to represent the fact that it could help you use the web (and by extension devices and services connected to it) hands-free.
Are there any overarching goals of Handsfree that drive design or implementation?
The goals change as myself and others discover new ways to use the library, but currently I’d like to complete #100DaysofCode where I use the library in a radically new way each day and then at the end create an interactive NFT called “Playing the web”.
The purpose for this goal is to demonstrate how easy the library is to use while also demonstrating its potential to help people make money hands-free.At the same time, the challenge of having to build one thing a day will hopefully improve the library by making it more succinct to use.
One much longer term goal is to position Handsfree.js in such a way that it becomes like a kind of “jQuery for Mixed Reality” or “jQuery for the next web”. I also have an ambition for the docs to be regarded as one of the most interactive and “best” docs out there.
What is the most challenging problem that’s been solved in Handsfree, so far (code links to any particularly interesting sections are welcomed)?
The plugin system! I’ve had to rewrite Handsfree.js from scratch 8 times to get this right. The plugin system was inspired by WordPress and frameworks like Vue and React, and it makes Handsfree.js so much easier to maintain.
My favorite thing about plugins is that anyone can create them without making Pull Requests, and anyone can use them without updating Handsfree.js! Then if you need extra specialized functionality you can npm install, link to, or just copy+paste the functionality!
The plugin system is what I’m most proud of, and I don’t think any other computer vision library supports plugins in this way. The really cool thing about plugins is that you can bulk enable/disable them by tag, for example: handsfree.enablePlugins(‘browser’) to use the browser hands-free...how cool is that!?
Are there any competitors or projects similar to Handsfree? If so, what were they lacking that made you consider building something new?
Handsfree.js has a Client Mode designed to incorporate its plugin system with any computer vision library, so I don’t like to think of them as competitors. Instead, here are some libraries Handsfree.js can augment:
ml5.js - This is the model for Handsfree.js, they have excellent documentation and a great community! They share many of the same models and an API to help you build your own
human.js - A newer library with a strong focus on human analysis, with more models than Handsfree.js currently
face-api.js and tracking.js - These are the OG’s in my opinion for multi-modal computer vision in the browser
What was the most surprising thing you learned while working on Handsfree?
It was very clear to me from the beginning that the project would become useful, but I didn’t realize how long it would take for me to convince other people 😅 It took me 3 years to get 500 stars and then only a few days to triple that!
How do you balance your work on open-source with your day job and other responsibilities?
Handsfree.js is my full time jam and has received significant support from:
The Studio for Creative Inquiry at Carnegie Mellon
Open Source Software Toolkits for the Arts (OSSTA)
The Clinic for Open Source Arts (COSA)
Glitch.com
Google PAIR
crowdfunding
and many others!
Do you think any of your projects do more harm than good?
Oh what an interesting question!
I believe that any open sourced project has the potential for doing harm and Handsfree.js is no exception. For example, imagine an ad company using the library to covertly, or even overtly, detect facial expressions to better serve ads. Or another company using it in a discriminatory way, like for job selection or as a requirement to access a service.
Another risk is that people deploy the library in risky environments without considering security (it is webcam driven after all), like using the library in HTTP sites instead of HTTPS or using it in a Browser Extension without considering which domain the webcam permission is given to (although I’ve published The Handsfree Browser Extension Starter Kit to address these issues).
What is the best way for a new developer to contribute to Handsfree?
The best way is to just start experimenting and creating things with it! Documenting, creating examples, writing tutorials, and even translating would be huge!
I think this is generally true of any open source project. The thing you use is typically only one part of a greater effort.
Where do you see software development in-general heading next?
This is the question that keeps me up at night!
First, I think that the mouse and keyboard will go out of style. I don’t think we will be using the mouse and keyboard much once AR becomes as ubiquitous as smart phones, and certainly not when Brain Computer Interfaces become practical.
Secondly, I think the future of software development will be much less about physically coding and more about verbally describing what you want. I think it’ll be more about conversing with a computer. There was a mind blowing GPT-3 demo last year that gives you a peek into this future.
Of course at some point I believe we’ll have super-intelligence, and I think the term “software” will become fuzzy if not archaic.
How do you plan to monetize Handsfree?
My dream is to become a tech “influencer” but by influencing my own work. The cool thing about working on an open source project like this is that there are a lot of content opportunities that can be monetized, like YouTube tutorials and online courses.