Console #22
Seaweed, Names, and Awful AI
GHunt
GHunt is an OSINT tool to extract information about someone’s Google account.
last commit: 9 hours ago, first commit: Oct 2, 2020
https://twitter.com/mxrchreborn
NasNas
NasNas is a new intuitive and user friendly C++ game framework.
last commit: yesterday, first commit: May 10, 2020
git-filter-repo
filter-repo is a much easier and faster alternative to filter-branch if you’re familiar with that.
last commit: July 27, 2020, first commit: Jan 29, 2019
Jupyter Notebooks Gallery
Jupyter Notebooks Gallery is an effort to archive the most interesting open-source Jupyter notebooks in one place.
Screego
Screego is multi user screen sharing via a web browser.
last commit: 1 hour ago, first commit: Sep 29, 2020
awful-ai
Awful AI is a curated list to track current scary usages of AI — hoping to raise awareness to its misuses in society.
last commit: July 29, 2020, first commit: Mar 27, 2018
nb
nb is a command line note-taking, bookmarking, archiving, and knowledge base application with:
plain-text data storage,
Git-backed versioning and syncing,
Pandoc-backed conversion,
global and local notebooks,
customizable color themes,
extensibility through plugins,
and more, all in a single portable, user-friendly script.
last commit: 14 hours ago, first commit: Nov 12, 2014
https://twitter.com/alphabetum
names.io
Names.io is a “global, exhaustive, scraped, name DB”.
last commit: 16 hours ago, first commit: Aug 18, 2020
https://twitter.com/KarmakarDebdut
seaweedfs
SeaweedFS is a simple and highly scalable distributed file system with an S3 API.
last commit: 8 hours ago, first commit: Nov 29, 2011
An Interview With mxrch of GHunt
What is your background?
I’m 20 years old and I’m from Alsace, France. I have never officially worked before, apart from a short period in freelance where I did some web development.
I learned how to program a few years ago, when I was in high school but learned everything at home from Codecademy when the Python 3 course was still free, and OpenClassrooms a little bit. However, the interactive courses like Codecademy were way more effective than OpenClassrooms.
I started with Python, because I wanted to learn the basics of programming, then I could apply these basics in other languages.
After learning Python, I learned Ruby at a bootcamp, in order to learn full-stack web development with Ruby on Rails. I really love Ruby because it makes web development clean (MVC), understandable, and easily “stackable”, but for script development I’m married to Python.After high school, in July 2018, I did a “Bac Pro Systèmes Électroniques et Numériques” but I didn’t like it at all. I stopped my studies because I didn’t learn, it was slow, and very time-consuming for nothing, and I needed to optimize my time. I did that web development bootcamp for 3 months in October 2018, this was my freelance/entrepreneur period, but my passion has always been cybersecurity since I was a kid. I discovered Hack The Box by chance and decided to go back to my roots, in October 2019.
From there, I searched for French people wanting to team up with me on Hack The Box, with the objective to progress together, and learn from the others etc.
This is where HideAndSec was born!
Then we speed ran all the boxes, trained every day and night, and I learned in 3 months more than my 3 highschool years. I decided to pass the OSCP in August 2020. Since then, I’ve been looking for a pen-tester position in a French-speaking country for a few months now.
Why was GHunt started?
There was an OSINT challenge on Hack The Box called “ID Exposed”. I won’t spoil it since it’s still active, but it has to deal with Google IDs, called “Gaia ID”.
Me and my colleague found Sector035’s articles on the subject and I was so impressed by all the information that could be found. I knew we could do even better by searching in the Google Photos metadata or by programming a little algo to guess the target’s location based on his Google Maps reviews.
Are there any overarching goals of GHunt that drive design or implementation?
Not really, I still have some ideas in mind I want to add that redditors gave me in the comments, but that’s all.
What were some of those ideas?
One of the more interesting ideas was to add Youtube comments. For example, fetching the last 3 YouTube comments, and maybe analyzing the aggressivity of the comments using the Jigsaw Perspective API.
Another suggestion was to get the phone model using the account recovery steps, which I'm working on right now.
What is the most challenging problem that’s been solved in GHunt so far?
Almost everything, but mostly trying to de-obfuscate the Javascript used in Google Maps to generate the internal API requests. I couldn’t de-obfuscate it, so I used selenium-wire which is a Selenium fork which includes a requests history.
So I automatized a browser to scrape all the reviews that are lazy-loaded then fetched the request history that contains the internal API responses, with weird JSON, and matched the reviews in the HTML with their location in the JSON.
Is GHunt intended to eventually be monetized if it isn’t monetized already?
Never. I mostly did this for the research, in order to make people aware about these “privacy issues” (not really, because the user can change settings for that), and for my portfolio to get a job!
What is the best way for a new developer to contribute to GHunt?
Take the Google Maps JS and de-obfuscate it to avoid using Selenium aha, same for Google Photos archives.
Honestly apart that I don’t know, I don’t think there is a lot more to do with it, once we include everything we can extract from a Google Account, there will not be anything more to do.
Where do you see GHunt heading next?
I don’t know, there is no direction to take. I just took it as a CTF challenge and I have fun with it (apart from when I’m getting issues everyday because of Selenium & Chromedriver).
Where do you see software development in general heading next?
Today there is software for almost everything, so I don’t really know. Maybe we’ll just improve existing software by implementing some technologies like AI or the blockchain. Today we already put a little bit of AI sauce everywhere.
Where do you see open source heading next?
I think more and more software will be open-source since it’s gaining importance and popularity, so it will boost other software. We create more and more good frameworks so people don’t need to learn all the basics to master these frameworks, so more and more people can contribute, learn, and create.
An Interview With Chris Lu of SeaweedFS
What is your background?
I’ve worked at Oracle, Composite Software, Salesforce, Facebook, and Uber. My background is in databases in general, federated queries, big data, and graph databases.
In school I majored in CS. I mostly code in Java for day jobs. I liked Ruby before. But the slow performance and the lack of ability to refactor moved me back to the strongly typed Java. I liked the first version of the play framework. But their move to Scala stopped me there.
I like Go because it is simple, performant, and easy to refactor. Everyone has limited time and brain cells. I want to use them to do actual things, rather than figuring out code tricks. Also, refactoring is a big deal. Let the computers and IDEs work as many checks as possible. Being able to freely refactor as fast as possible is super important to adapt to changes.
Why was SeaweedFS started?
SeaweedFS was started as a side project after working with Eric Xing, FeiFei Li, and Kai Yu on an AI startup idea on image recognition around 2011, which needed to process a lot of images.
At that time, only Facebook’s Haystack paper showed good support for the large number of image files. However, it was just a paper. I waited a couple of years, but still no open source options. So I started to implement it as a distributed object store in Go.
The object store has been stable for a few years. I thought there would be someone to pick it up and build more stuff on top of it. For example, maybe a new file system, a message queue system, or even a database. However, no one cared. Most users just want to pick a solution and use it, same as me waiting for a Haystack implementation.
So I started to implement the SeaweedFS file system layer. Lacking working experience and knowledge with common storage systems, I chose to build the system as scalable as what I thought it should be.
Are there any overarching goals of SeaweedFS that drive design or implementation?
The overarching goal for SeaweedFS was to make data access fast, for both reads and writes, and for both structured and unstructured data.
What tradeoffs have been made in SeaweedFS as a consequence of these goals?
The goal has to be delivered in phases. Big dreams are built upon layers. A full solution is hard to achieve at the beginning. For open source projects, the wider the use case, the better. Only with more users can open source projects sustain themselves.
What is the most challenging problem that’s been solved in SeaweedFS so far?
Active-Active cross cluster synchronization.
Transparent tiering to cloud storage.
How do you intend to eventually monetize SeaweedFS?
A big company hosted SeaweedFS on cloud to be faster with dedicated servers and cheaper without access fees. I think a hosted SeaweedFS can be provided as a service.
What is the best way for a new developer to contribute to SeaweedFS?
Just start using it. The ideas will come to you when using SeaweedFS. There are many ideas yet to be implemented.
Where do you see SeaweedFS heading next?
SeaweedFS plans to support structured data storage and access.
Where do you see software development in general heading next?
Software is going more complicated. With more components to add, it is better to separate the components into its own containers. Kubernetes is becoming a standard way to deliver a fully operational distributed system. A software engineer needs to learn not only how to develop code, but also how to manage containers as dev ops, or even codify the operations.
Where do you see open source heading next?
Open source is mostly for collaborating with other techies.
There isn’t much secret sauce in software. With projects getting more complicated, companies can choose to open up their source code and not be afraid of being copied.
What are the other projects you built and why they are not as successful?
I used to work on a closed-source database search software. It was ok, with paying customers, including eBay, IBM, Costco, European Central Bank, etc. However, it is hard to scale with closed-source and with narrow scope. With open source projects, it is easier to benefit a larger audience, getting a better feedback loop. The wrong direction is more dangerous than failed attempts in the right direction.
Like what you saw here? Why not share it?
Or, better yet, share Console!
Also, don’t forget to subscribe to get a list of new open source projects curated by an Amazon software engineer directly in your email every week.