Latent Space

Real-time collaboration during the COVID-19 pandemic is hard. Without being able to meet in person, people are forced to connect via video call platforms such as Zoom, Google Meet, and Jitsi. This problem is compounded when dealing with music. Today’s leading video call platforms have extremely high latencies; while humans can easily recognize latencies of 10ms and more, Zoom’s latency is 135ms, Google Meet’s latency is 100ms, and Jitsi’s latency is 500ms. Latent Space is a decentralized video calling web app that utilizes recurrent neural autoencoders with embeddings specifically for music to transfer encoded (compressed) audio files via RTC packets with minimal latency. Musicians and anyone wishing to meet virtually can join a call at latent-space.herokuapp.com, where they can then record audio accurately and with ease.
Won 3rd Place Overall at HackMIT 2020, MIT's largest undergraduate hackathon and one of the world's largest collegiate hackathons. Featured on Medium.

Tools: HTML, CSS, JS, Python, TensorFlow, jQuery, NodeJS, WebRTC, Web Audio API, TURN, ICE, STUN, Twilio, Heroku, SciPy, PyDub, NumPy, Google Cloud, TPUs

GitHub Website Video Demo Finalist Presentation Sample Video Call Preprints.org Paper

Close

Centivize

Centivize is a blockchain/AI based progressive web app that allows users with conditions less critical than COVID-19 to obtain diagnoses and treatment plans based on their medical histories and post geofenced requests that volunteers are incentivized to fulfill via cryptocurrency. We use an APIMedic-based smart diagnosis algorithm that converts a user’s medical history (inputted on the platform or via Alexa/Google Home) into a likely diagnosis and multi-paragraph treatment plan shortened into a list of basic treatment necessities by sentence transformer ML model based on NLTK and RoBERTa. Users can post treatment plans or other requests on the Centivize dashboard for volunteers to fulfill in exchange for crowdfunded cryptocurrency based on community upvotes and executed via Ethereum smart contracts.
Won 3rd Place Overall at COVIDathon 2020, the world's largest decentralized AI workplace hackathon. Awarded 40,000 OCEAN tokens (~$75,000). Worked with the Platform Team at SingularityNET to publish Centivize's summary and similarity algorithms on SNET's dApp: available here. Centivize will appear on Ocean Protocol’s “Compute-to-Data” platform in mid-2022.

Tools: ReactJS, HTML, CSS, JS, Voiceflow, Google Cloud/Maps, Alexa/Google Home, RadarJS, Python, Transformers, Ethereum

Website Telehealth App GitHub Devpost Video Demo Ocean Protocol Proposal

Close

Topdoc

Topdoc is a 501c3 nonprofit organization that utilizes artificial intelligence to provide free, end-to-end medical screening in under-resourced communities. Aimed at democratizing access to healthcare, we offer a variety of free screening algorithms for a variety of diseases, including glaucoma, skin cancer, tuberculosis, and brain cancer. We also built an online forum where a panel of online doctors can offer treatment plans, providing patients with low-income alternatives to in-clinic screening and treatment while also augmenting their customer bases. Topdoc is currently working with Dr. Derrick Koo at Vista Eye Center to implement a deep-learning-based early diagnosis glaucoma screening CNN that operates on retinal eye scans (first algorithm that not only operates on images but also considers variables such as corneal thickness, eye pressure, patient age, etc.). This algorithm will soon be integrated into routine patient checkups at Vista Eye Center.
Skin cancer algorithm won 3rd Place Overall at QuestHacks 2019.

Tools: Python, TensorFlow, echoAR, PyTorch, scikit-learn, Keras, Flask, C++, OpenCV, Pandas

Website GitHub Guide to Services

Close

Rize

Rize is a mobile app aimed at facilitating telemedicine. With Rize's smooth and user-friendly scheduler, patients can set up appointments with local doctors quickly and with ease, and all meetings are displayed with virtual meeting links in the Rize calendar. Upon joining a virtual meeting with a medical professional, Rize automatically records the conversation and processes audio real-time; based on user time interval settings, Rize uses NLTK and sentence transformers to output a real-time summary of the audio transcription that has been recorded in the most recent time interval, which doctors and patients can use to understand the key points of the meeting. Rize also auto-fills EHR records to help doctors maintain medical records.
Tools: Swift, Xcode, Python, PyTorch, NLTK, transformers, RoBERTa, Core ML, Java, NodeJS, Firebase, Twilio

GitHub

Close

Tickbird

Tickbird is a Swift/Node.js mobile app based on the TesseractOCR CNN framework allowing visually impaired patients to aurally understand their prescriptions or the labels on their pill bottles in order to gain independence and avoid the prospect of lethal miscommunication regarding necessary medicines from their doctors.
Top 10 Overall and 2nd Best Mobile App out of 35+ teams at OmniHacks 2019. Acquired 60+ downloads and 1.5K+ impressions on the iOS App Store. Featured in the Saratoga Falcon.

Tools: Swift, Java, Xcode, NodeJS, Tesseract, Firebase

Landing Page iOS App Store GitHub Video Demo

Close

Archiscape

Archiscape is a progressive web application (PWA) aimed at revolutionizing the fields of smart infrastructure, urban planning, and home design. Construction workers are burdened by the long and tedious task of manually creating 3D models from 2D blueprints. Our ThreeJS/BFS/ML model streamlines this process; once a user submits a floor plan PNG, they receive a responsive, interactive 3D model of the design they provided. These workers are not the only ones facing problems in the real estate industry. To facilitate safe and secure tourism (especially with the rising severity of COVID-19), realtors, buyers, and inspectors are all looking for ways to create and view virtual house tours without the hassle of setting up a 3D camera. Archiscape allows users to input regular images of rooms in their house, and our neural network (along with pano stretch augmentation techniques) converts those images into an editable 3D virtual tour.
Won 1st Place Overall at LAHacks 2020, SoCal's largest hackathon and one of the world's largest collegiate hackathons. Featured in CBS News, The UCLA Newsroom, Build for COVID-19, Scommerce, and The Saratoga Falcon.

Tools: AngularJS, BFS, Firebase, Google Cloud, HTML, CSS, JS, Python, Three.js

Website GitHub Devpost Video Demo

Close

AWS DeepRacer

As founder of the DeepRacer Palo Alto team, I have had the thrilling opportunity to explore reinforcement learning (RL) with my peers in the context of building self-driving miniature racecars—components of which must be programmed on the spot—at AWS-hosted competitions across the Bay Area. I partnered with StreetCode Academy to host RL workshops for high school students new to AI/ML.
Won $150 First Place Award at DeepRacer February 2020 Menlo Park Competition

Tools: Python, AWS, Reinforcement Learning

DeepRacer Website GitHub

Close

TogaLink

TogaLink is an advanced web application that combines geolocation with healthcare to allow senior citizens to request essential items that volunteers can then deliver. Partnered with Mayor Howard Miller and West Valley Community Services (WVCS) Executive Director Josh Selo to use TogaLink as a means of digitizing the food delivery and volunteering processes currently in use by WVCS members. In use by the Counties of Santa Clara, San Mateo, and Santa Cruz.
Tools: HTML, CSS, JS, Google Cloud, Disqus, Google Maps, Svelte, Sapper, Vue.js

Website GitHub

Close

The Official SHS Trophy App

Chosen by SHS administration to develop an Android application displaying school trophies; praised by athletics department and visible on a kiosk in the gymnasium. Paid $1.8K for our services to the school, earned $1K for App Development Club, and donated $300 to Black Lives Matter.
Tools: Android Studio, Java, XML, Figma

GitHub

Close
Vignav Ramesh

Hi, I’m Vignav Ramesh.

I'm an undergrad studying computer science at Harvard. Broadly, my academic interests center around natural language processing, multimodal learning, and mechanistic interpretability.
I'm currently a research scientist at Harvard Medical School, where I am co-advised by Profs. Pranav Rajpurkar (HMS DBMI) and Andrew Ng (Stanford CS). Previously, I built Arc, a tooling layer for enterprises to provision teams of specialized LLM agents (deferred $1M to continue undergrad).
I have 5 years of software experience, 10+ hackathon wins, and have published several papers in top-tier AI/ML venues. In my free time, I enjoy playing tennis and practicing the double bass.
Reach out at vignavramesh [at] college [dot] harvard [dot] edu to chat!
Press: Geekwire (x2), Mercury News, CA Times, East Bay Times, CBS News, Business.org, AP News, Central Charts, Markets Insider, Morningstar, GlobeNewswire, KFM BFM, Euroinvestor, OneNewsPage, MarketsAsk, CompuServe

Work & Research Experience

Research Scientist, Medical AI Lab | HMS DBMI

  • Developed GPT-3/BERT-based NER models (1M Huggingface downloads) to remove references to priors in chest X-ray radiology reports (work cited by Google & Meta AI)
  • Currently exploring unsupervised domain adaptation and co-training methods to develop foundation models for zero-shot medical image segmentation

Research Intern, Stanford University Laboratory of Quantitative Imaging and Artificial Intelligence

  • Developed a SOTA contrastive learning framework using neural style transfer for detection and classification of referrable vs. non-referrable diabetic retinopathy with up to 90% label reduction
  • Previously trained SOTA architectures for COVID-19 lung lesion segmentation on CXRs

Research and Development Intern, SingularityNET Foundation

  • Built explainable & unsupervised natural language generation, text segmentation, and question answering systems to enhance the general conversational intelligence of proto-AGI pipelines; all models matched or outperformed SOTA benchmarks with zero supervised training.

🤖

Founder & CEO, Arc

  • Built custom infrastructure for agent auto-specialization, synthetic dataset generation, and multi-agent orchestration
  • Piloting tech w/ Alchemy (Series C1 web3 development platform); partnered with Superpowered AI (YC S23) to develop SOTA retrieval-augmented generation systems

🔊

Cofounder & Co-CEO, Speakeasy

  • AI startup providing automated accent training services for enterprise
  • Built first AI model that measures mouth/tongue position purely from user audio; signed LOIs w/ top hospitality brands (Four Seasons, Marriott, Hyatt, Fairmont, IHG, etc.)


Books


Publications



Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors [paper, code, dataset]

Vignav Ramesh*, Nathan A. Chi*, Pranav Rajpurkar

NeurIPS 2022, SAIL 2023


Contrastive learning-based pre-training improves representation and transferability of diabetic retinopathy classification models [paper, code]

Minhaj Alam, Rikiya Yamashita, Vignav Ramesh, Tejas Prabhune, Jennifer Lim, R.V.P. Chan, Joelle Hallak, Theodore Leng, DL Rubin

Nature Scientific Reports


COVID-19 Lung Lesion Segmentation Using a Sparsely Supervised Mask R-CNN on Chest X-rays Automatically Computed from Volumetric CTs [paper, code]

Vignav Ramesh, Blaine Rister, Daniel L. Rubin

SIIM 2021 (Oral)


Unsupervised Tokenization Learning [paper, code]

Anton Kolonin, Vignav Ramesh

EMNLP 2022 (Oral)


Context-Driven Question Answering Based on Link Grammar [paper, code]

Vignav Ramesh, Anton Kolonin

AGI 2021 (Oral), Hosted INLP Workshop 2021-2023 (youngest workshop host in history)


Interpretable Natural Language Segmentation Using Link Grammar [paper, code]

Vignav Ramesh, Anton Kolonin

SAIence 2020 (Spotlight)


Natural Language Generation Using Link Grammar for General Conversational Intelligence [paper, code]

Vignav Ramesh, Anton Kolonin

MLNLP 2021

Patents



[US #17/447,508] SYSTEMS AND METHODS FOR EVALUATING GAME ELEMENTS (Nonprovisional, pending)


[US #63/156,357] UNSUPERVISED AND EXPLAINABLE NATURAL LANGUAGE GENERATION USING LINK GRAMMAR FOR GENERAL CONVERSATIONAL INTELLIGENCE (Provisional, issued 3/3/21)


[US #63/156,359] INTERPRETABLE NATURAL LANGUAGE SEGMENTATION BASED ON LINK GRAMMAR (Provisional, issued 3/3/21)

Education

Harvard University

BA/MS, Computer Science • GPA: 4.0/4.0
John Harvard Scholar, Detur Book Prize Recipient

Selected Coursework: Deep Learning*^, Advances in Computer Vision*^, Embodied Intelligence*^, Large Language Models*^, Reinforcement Learning, Interpretable ML*, Convex Optimization, Data Structures & Algorithms, Advanced Topics in Data Science, Linear Algebra & Real Analysis (*Graduate-level, ^Cross-registered at MIT)

Organizations: Chief Consulting Officer @ Harvard Data Analytics Group, Sourcing Principal @ Harvard Undergraduate Capital Partners (<1% acceptance rate)


Service Leadership

Founder and Double Bassist, Raven

February 2019 - August 2022

The Raven Ensemble is a nonprofit organization with the mission “play to provide.” We play famous classical pieces with a unique, childlike flair in order to raise money to allow students in under-resourced communities to obtain an education and follow their entrepreneurial dreams. We performed our album Virtuoso in multiple retirement homes across two states, opened the ears of more than 300 listeners, and raised over 500 dollars for the nonprofit organizations Lemonade Day and Pencils of Promise

Kidpreneurship Founder and Lead Teacher, StreetCode Academy

January 2020 - June 2022

At StreetCode, I founded the Kidpreneurship class to empower kids to view their age as a strength, not a weakness, and capitalize on their youth while pursuing entrepreneurship in adult-dominated industries. Previously, I volunteed as Lead Teacher of Game Design II and as a mentor for the Intro to Code class. I also built the Wantrepreneur platform to foster kidpreneurship in the StreetCode community.

Founder, President, and Lead Instructor, CoderDojo Saratoga

December 2019 - September 2021

CoderDojo Saratoga is Saratoga’s first free, volunteer-led programming club tailored specifically to Saratoga's elementary and middle schoolers. As founder and president, I organized a team of 10 mentors to serve 130+ students across 2 schools.

Projects

Selected Awards & Honors

2021 Helen and Paul Chang Foundation New Investigator Travel Award Recipient

Awarded $1250 by SIIM for my research on COVID-19 severity quantification
Youngest recipient in history

2022 U.S. Presidential Scholar Semifinalist

Named a Presidential Scholar candidate by the White House Commission on Presidential Scholars and the U.S. Department of Education (top 0.00017% of U.S. high schoolers)

2022 Coca-Cola Scholars Regional Finalist

One of 250 Regional Finalists selected from over 68,000 applicants on the basis of leadership, academics, and dedication to community
Previously selected as one of 1,617 Semifinalists

2021 Rensselaer Medalist

One of the top 20 juniors at Saratoga High School selected to receive a prestigious award for the annual Junior Awards Ceremony
Elected by the Math/Science department to receive the Rensselaer Medal, which recognizes a student's strength and success in math and science


OpenCog Foundation Prize for Best Student Paper @ AGI-21

Awarded the $250 Best Student Paper prize by the Artificial General Intelligence Society at the AGI-21 conference

Featured Paper

One of two featured and successfully reproduced papers out of 100+ submissions at the 2020 S.A.I.ence Conference

Gold Division Programmer

USA Computing Olympiad (USACO)

AIME Qualifier

Awarded a Certificate of Distinction


Latent Space: 3rd Place Overall

HackMIT, MIT's largest hackathon and one of the largest collegiate hackathons in the country

Centivize: 3rd Place Overall

COVIDathon 2020, the world’s largest decentralized AI hackathon
Won 40,000 OCEAN tokens (~$75,000)

Archiscape: 1st Place Overall

LA Hacks 2020, Southern California’s largest collegiate hackathon
Featured in CBS News, The UCLA Newsroom, Build for COVID-19, Scommerce, and The Saratoga Falcon

Skills

Programming Languages & Tools

Areas of Expertise

  • Natural Language Processing
  • Computer Vision
  • Full Stack Development

© Vignav Ramesh 2020.