HOME > PROJECTS

... a collection of past and present work and projects


"SPARTICLE": AN iPHONE GAME
Independent project - Spring 2009
Sparticle is an iPhone game that make use of the device's accelerometers, openGL, openAL and a bunch of physics.
[more info]


"CHALKY": AN iPHONE / ANDROID APP
Independent project - Summer 2008, Winter 2009-10
iChalky is a mobile application that makes use of the device's accelerometers, openGL and microphone in an attempt to illustrate the fact that humble perceptual measurements and simple constraints set in a physically meaningful environment can lead to interesting and sometime unexpected behavior.
[more info]


VU.iT: A SMART PARCEL INDUCTION DEVICE
Work commissioned by Postea, Inc. - 2007-2008
Postea is an investment company focused on postal innovation and technology. This project consisted of a research and development effort leading to two working prototypes of a smart scale device that determines a parcel's weight, 3D dimensions, performs OCR address and barcode parsing in a single step. Such a device is particularly relevant at a time when traditional postal services are trying to move towards dimensional pricing and offer cost efficient track and trace services.
[more info]


"PHASE": YOUR MUSIC IS THE GAME
Work commissioned by Harmonix Music Systems, Inc. - 2007
The music analysis algorithms developed for this client under the MUSICAL TRANSIENTS ESTIMATION thread were integrated within iTunes for an iPod game entitled "PHASE" and released in the fall 2007.

Phase is an interactive music game for the latest Nano, Classic and 5th generation iPod® that lets you experience songs in an entirely new and unique way. Choose a song in your iTunes® library, and Phase will turn it into a playable game level. Press and glide the iPod's® click wheel along with the rhythm of the music to catch notes and flowing sweeps. Travel through lush landscapes while playing your favorite music. In Phase, your music is the game!
[more info]

back to top


DIGITAL AUDIO WATERMARK SHAREWARE
Independent project - 2006-present
This shareware is intended to target small to medium size applications that may require an audio watermark. This audio watermarking system is based on Spread Spectrum signaling techniques, a communication concept that has been widely used throughout at least three decades. Its application to embedding data within multimedia content can be traced back to academic papers published in the early 90s.

The EyM Audio Watermarking tools are shareware utilities that are offered "as is". They to embed (resp. retrieve) a short data payload within (resp. from) 16-bit PCM audio wav files.
[more info]

back to top


3D TAGS TRACKING
Work commissioned by Oblong Industries, Inc - 2006-present
Oblong Industries, Inc. is the first commercially oriented gesture recognition company. It licenses its proprietary human gesture-based operating system to qualified licensee/development partners in various industries. Through its breakthrough human/machine technology, Oblong achieves superior, intuitive control and improved efficiency over complex systems increasingly overburdened with data, images and tasks.

The typical setup uses a set of cameras and matching hardware that can track the 3D positions of elementary targets such as small balls. With the help of gloves that were judiciously marked with such elementary targets, the system subsequently infers the user's hand and finger positions and orientations. These are finally parsed by Oblong's proprietary gesture recognition engine ("g-speak") where they become interpreted and used to control a running application.

The glove tracking stage was originally handled through a commercial motion tracking software system that was not truly intended to perform in this manner. The project consisted of the design, development and implementation of an algorithmic alternative to this component - a system that would be under Oblong's complete control and tailored to the specific needs of the company's gestural interface. The delivered system locates known tags from a noisy cloud to dot positions in space and recovers their positions and orientations with high accuracy, reliability and efficiency.

back to top


FRAUD AND ABUSE DETECTION IN GRAPHICS
Study commissioned by MarkMonitor, Inc. - 2005-2007
MarkMonitor is in the business of tracking fraudulent or illegal uses of corporate brands appearing on the Internet or in "phishing" emails. Embedding malicious offers within graphics is a growing trend among con artists trying to circumvent existing text-based filters. In order to address this growing threat the client needed a graphics recognition engine capable of locating and recognizing known "toxic" graphical objects such as words or logos within arbitrary image data.

This project consisted of the design, development and implementation of suitable image processing algorithms, packaged into a compact and portable API to facilitate their integration within the client's system architecture. The first complete system delivered in Summer 2005.

back to top


1D BARCODE DECODING FROM IMAGE DATA
Independent project - 2004-present
Design, development and implementation of a series of image processing algorithms aiming to read optical scans of 1D barcodes despite potentially crude image resolution. These algorithms were packaged into a simple to use ActiveX control (OCX). This component is aware of the following symbologies and encoding systems: EAN-13, UPC-A, JAN-13, ISSN, ISBN (a.k.a Bookland), ISMN, EAN-8, UPC-E, Code 128, UCC/EAN-128, Code 39, Code 93, Interleaved 2 of 5, supplemental 2 and 5 digit codes, Codabar (a.k.a Ames Code/USD-4/NW-7/2 of 7), Patch codes (1, 2, 3, 4, 6 and T).

This component is available on a "one time fee" basis (i.e. free runtime) and a free demo application turning you webcam into a barcode reader is also available.
[more info]

back to top


TRANSACTION RISK/ACTUARIAL MODEL
Study commissioned by Escher Group Ltd. - Fall 2004
Checking a customer's credentials and seeking authorizations from central services is key to preventing casual frauds when offering various types of transactions. Unfortunately, this type of risk control process will typically incur an additional upfront variable cost to the transaction. In the end, the total upfront cost of performing such control might not be worth potential risks associated with this transaction.

This study suggested a means for an organization to infer its own actuarial model from its past activity, much like an insurance provider would. It showed how such model may provide a measure of risk premiums associated with any transaction. Finally, it suggested a means to use such risk premium estimations in order to decide whether or not to incur the variable cost associated with risk control when facing a new transaction request. Through numerous simulations it showed that the resulting bottom line was systematically improved when these risk premiums were taken into account.

back to top


NETWORK DATA TRANSPORT OPTIMIZATION
Study commissioned by Escher Group Ltd. - 2004-2006
Escher Group Ltd. is the developer and provider of a peer-to-peer messaging solution known as Riposte® or WebRiposte®, and based on its proprietary Message Server technology. Its asynchronous message-based data transfer is built on top of the User Datagram Protocol (UDP). This study derived probabilistic models for this protocol and for a more traditional Transmission Control Protocol (TCP) in order to optimize both provided with arbitrary network characteristics and compare their respective maximum performance. This exercise served to show that Escher's UDP-based protocol systematically outperformed a TCP data transfer for peer-to-peer data distribution. It also highlighted the technologies ability to get the best out of virtually any type of network.

Follow up studies since 2004 have resulted in an elegant and distributed approach to congestion control, throughput optimization and response time for online transactions through Escher's Message Server technology.

back to top


NIKE OPTICAL CODING SCHEME
Work commissioned by NIKE, Inc. - Spring 2004
Design of a proprietary graphic symbology conveying a unique identifier and applicable to a wide variety of products. This work included the design, development and implementation of a series of algorithms aiming to retrieve these identifiers when the product is casually presented to a commercial grade optical scanner such as a webcam or similar device, coping with a range of geometrical distortions.


LAPAROSCOPIC TRAINER
Work commissioned by Blackdust Design - 2004, 2006
The LTS2000, developed by Harrith Hasson, MD and Blackdust Design, is an interactive sensing module for laparoscopic skill training and measurement. It consists of a rotating sensor carousel with several coordination and knot tying tests. It provides feedback via video overlay and a software front-end. It was demonstrated at the SLS2002, AAGL2002, and MMVR2003 conferences. Eric's contribution consisted of the development of a robust intuitive and elegant software front-end providing real-time animated and audio feedback in addition of progress reporting and logging.
[more info]


"EYETOY: ANTIGRAV" MACHINE VISION / TRACKING SYSTEM
Work commissioned by Harmonix Music Systems, Inc. - Winter 2003/04
Development and implementation of a vision system in the context of a Sony PlayStation/2 video game that uses a player's physical gestures and postures as character controls. The system's specs included face tracking, hand tracking and shape recognition using Sony's USB camera ("Eyetoy").
EyeToy: AntiGrav was unveiled at the 2004 E3 conference in Los Angeles, where is won the Best of E3 2004 IGN.com Award in the "Most Innovative Design" category and was nominated for the Best of E3 2004 Game Critics Award in the "Best Puzzle/Trivia/Parlor Game" category. It is published by Sony Computer Entertainment and started shipping in November 2004. It subsequently ranked No.4 in the Today Show's 2004 "Top Video Games" toy contest, where it was chosen out of 83 new games across all platforms and tested by 10,000 kids.

[more info]


MUSICAL TRANSIENTS ESTIMATION
Work commissioned by Harmonix Music Systems, Inc. - 2003, 2006
Research and development of a means to detect, locate and characterize musical transients from an arbitrary audio stream. A series of algorithms that infer an audio stream's rhythmic structures, locate musical transients and characterize their strength and spectral properties were designed, developed and implemented.

The original solution was delivered in the Summer of 2003 and later revised in the Spring of 2006 to further extract higher level musical information such as tempo and beats.

back to top


ACODE: A REGISTRATION-FREE SYMBOLOGY WITH POSITIONING FEEDBACK
Work performed while Director of Strategic Research at Escher Group Ltd. - 2000-03
Designed, developed and implemented a novel technique for encoding data into and decoding data from a coded pattern using an angular symbology in which data symbols are represented by angular orientations of data modulation patterns relative to a reference modulation pattern.

The modulation patterns are selected to have components that are localized in a Fourier transform domain and the technique may convey arbitrary digital data within 2D images. This symbology's targeted regime of operation would typically involve D/A and A/D stages: a coded image is printed on some physical medium, and subsequently optically re-digitized prior to the retrieval of the embedded digital data. The technique was explicitly designed to minimize any registration issue, which typically occurs for this type of technology. It was also designed to be graphically versatile and sensitive to the appearance of the resulting coded images. Additionally, it exhibits the unique ability to convey positioning information throughout the image that carries it. Not only can the data payload be retrieved from any section of the coded printout, but the position of the imager with respect to the printout can also be estimated.
[more info]

back to top


FIBER-FINGERPRINTING
Work performed while Director of Strategic Research at Escher Group Ltd. - 2000-03
FiberFingerprint analysis uses the naturally occurring irregularities of paper textures in order to provide a reliable means to discriminate between various pieces of paper. The algorithms make use of registration marks as means to identify the area of the medium where the analysis should take place. These registration marks typically consist of a few small dots, spanning a total surface area of typically less that 10 mm2. The practical motivation for this technology is security. Compared to a barcode, digital watermark, or other embedded serial number, the identity of a FiberFingerprinted object is difficult to forge, given the length scale and three-dimensional aspects of the physical properties being sampled.

This work required the design, development and implementation of numerous image-processing algorithms for the conditioning of the fingerprints and for precise image registration. Along the way, several numerical and theoretical models were built to characterize and extrapolate the technology's verification (one-to-one) and identification (one-to-many) performances.
[more info]

back to top


THE STICKSTERZ: AN AUDIO PLAYER VISUALIZATION
Independent project performed out of curiosity - Fall 2000
This free visualization for Real Networks and Windows Media Players showcases dancing stick figures that improvise their choreography based on simple audio transients and simulated physical constraints. Rather than scripting dance moves, the figures decide on the fly which one of their degrees of freedom (or "joints") they'll move and by what amount. Simulated inertia and gravity take care of the rest. Forcing additional constraints among their "joints" varies the nature and feel of their improvised choreography. The minimal 3D rendering and dynamics were written from scratch (ANSI C).

Since their first release on Halloween 2000, there has been an estimated excess of 40,000 downloads.
[more info / get your own copy here]

back to top


DAFX98: MUSICAL GESTURES AND AUDIO EFFECTS PROCESSING
Independent research project presented at the Digital Audio Effects Processing - 1998
The first annual Digital Audio Effects Processing conference (DAFX98) took place in the beautiful city of Barcelona. The call for paper was an opportunity for me to revisit some toy ideas that had emerged from my work at the Media Lab. It uses the concept of Musical Gestures (which came from my Ph.D. dissertation) in the context of audio effects processing. As an embodiment of the general framework is presented, I brought with me a sound example resulting from the processing of a recording of Ravel's Sonate pour violon et violoncelle. Harmonic structure likelihoods (standing for the chosen set of musical gestures for this example) were estimated from the polyphonic recording and they were then used to add a synthetic layer consisting of female voices (generated as part of the processing through some trivial wavetable procedure). The resulting audio is satisfying and it illustrates the fact that even a difficult problem such as real-time polyphonic tracking can be reasonably achieved with a soft analysis/control system which doesn’t attempt to capture high level musical intentions but rather confines itself to an expressive and humble set of measurements.
[more info]


MUSICODE: A COMMERCIALLY DEPLOYED AUDIO WATERMARKING TECHNOLOGY
Work performed while Research Staff Engineer at Verance Corporation (formally ARIS Technologies, Inc.) - 1997-2000
Eric is a Co-inventor of MusiCode®, Verance's core audio watermarking technology, which was selected as the worldwide industry standard for DVD Audio copy control and for the Secure Digital Music Initiative (SDMI) in 1999. This patented solution that has met every challenge thrown at it through technology bakeoffs, bench marks and high profile listening tests.

The technology is also at the center of the company's ConfirMediaTM offering, which currently monitors the 100 top U.S. media markets plus all major TV and Cable networks, reaching 85% of the U.S. viewing and listening audience.
[more info]

back to top


EMBEDDING MODELING
Ph.D. research work performed at the MIT Media Laboratory - 1992-1996
Documents (Pdf files):
"Pitch Synchronous Embedding Synthesis" (Ph.D. dissertation - 1996)
"Stochasticity versus Determinism for linear and non linear signal modeling" (Ph.D. qualifying study - 1995)
"Recursive Estimation of Global Models for Embedding Surfaces" (Ph.D. qualifying study - 1995)

There is a clear distinction between modeling a system from observed data and measuring a specific set of features from the data set. Ideally, modeling should be approached without any pre-conception about the system's architecture. The training data should stand for the unique relevant source of information from which our task is to derive as much knowledge and understanding about the system's mechanism as possible. Conversely, measuring a specific feature from input data implies the prior knowledge of its nature. Ironically though, these two tasks are traditionally so closely related that their distinction resides only in their purposes and not all that much in their implementation or mechanism.

Until recently, linear system theory was the only modeling tool available and its extensive use made us forget about the strong assumptions it relies upon. The notion of a deterministic (or predictable) process that was introduced by Wold's decomposition characterizes only a subclass of deterministic systems: deterministic and linear systems (the future is a linear combination of the past). Through this decomposition a process that may appear non-deterministic or even purely non-deterministic, is not guaranteed to be stochastic at all. It might be the chaotic output of a non-linear deterministic dynamical system. The estimates of the second order statistics of a deterministic, but chaotic, system can be amazingly similar to the ones of a random white noise.

Inferring non-linear models from observed data without any pre-conception concerning the architecture of an eventual model is no longer a dream. Floris Takens' 1981 Embedding theorem can be applied to time-series and lead to a general scheme for the inference of physically meaningful models from observed behaviors.

Modeling a time series as a non-linear hyper-surface in a lag-space was the inspiration behind Cluster-Based Embedding Modeling, which has since been the object of a patent and a letter to Nature.

back to top


INTERACTIVE MUSIC AND MACHINE LISTENING
Work performed as a Research Assistant at the MIT Media Laboratory - 1992-1996
Eric contributed to the design and development of various interactive musical systems (HyperInstruments, BrainOpera) in Prof. Tod Machover's Music and Media group, principally through the design and development of a variety of real-time audio signal processing algorithms.