← SensorSynthFM
Desk Research · Music Technology2026Active Build

SensorSynthFM:
Desk Research

FM SynthesisExpressive TouchDMI MappingPassive SensingAUv3 WorkflowCognitive Load

What This Research Can and Cannot Claim

The research is useful because it narrows the build question. It is not a launch argument. App listings, academic papers, product pages, forum threads, and trade coverage can show adjacent products and design constraints. They cannot prove that musicians want this exact instrument.

Supported

FM, touch expression, host workflows, and mapping literature all point to a credible design space.

Not proven

No source here validates demand, product-market fit, accessibility outcomes, or finished prototype quality.

Build implication

The first demo should answer one interaction question clearly before adding product breadth.

The Working Question

SensorSynthFM is best explained as a test of whether FM synthesis becomes more playable when sensors, touch, and visual feedback replace a dense parameter editor as the main performance surface. The instrument should not ask a performer to manage a routing spreadsheet while trying to play.

The strongest version of the project is not simply “FM on iPad.” That already exists in several forms. The stronger question is whether the iPad can make FM feel more physical: tilt changes timbre, proximity or light shifts modulation, touch gives intent, and the screen explains what changed before the sound feels arbitrary.

What the Source Set Says

1. The test

SensorSynthFM is not trying to prove a market. It is trying to test whether FM synthesis becomes more playable when a few mapped sensors, touch actions, and visual cues replace a dense editor as the performance surface.

2. The landscape

The research found capable iPad FM apps, capable expressive instruments, and capable hosts. It did not find a shipping iPad instrument whose main idea is sensor-driven FM performance.

3. FM on iPad

FM gives the project a sharp sound identity, but it also creates the interface problem. A small demo should not dump the operator matrix onto the performer.

4. Touch expression

GeoShred, Animoog Z, Orphion, and MPE practice show that iPad performance can be serious. SensorSynthFM should borrow that seriousness without pretending it already matches those instruments.

5. Workflow

A future product has to fit AUv3, looping, host recall, and tempo-sync realities. The thesis demo should still keep sequencing secondary until the sensor-to-FM relationship works.

6. Passive sensing

The sensor layer is the reason for the project to exist on iPad. It has to feel bounded and readable, not like random environmental noise wearing a musical name.

7. Visual legibility

The screen has to show which sensor is active, what it is changing, and how strongly it is changing it. If the mapping cannot be explained fast, it should be cut for the demo.

8. Cognitive load

The best version is deep enough to reward practice and simple enough that the first cause-and-effect relationship is obvious.

What This Changes for the Thesis Demo

The immediate target is not a bigger synth. It is a smaller, clearer proof: sensor input changes FM timbre, the screen explains the change, and the result survives a short musical demo.

  • Build one integrated screen before chasing a larger product surface.
  • Make two or three sensor-to-FM mappings audible, visible, and explainable.
  • Treat preset browsing, full MPE architecture, integrated sequencing, and store readiness as later work.
  • Use the demo to find which mappings are musically defensible and which ones are only interesting on paper.

Selected Source Trail

The private research corpus is larger than this first public list. Some material is prototype notes, private documents, implementation notes, and working documentation. The links below are the short argument trail: enough to let a reader walk the same argument without exposing private project records.

App Store listings and product pages can change, so this page treats them as current evidence to recheck before publication, not permanent facts. The broader source atlas follows this selected trail.

FM theory and iPad FM

These sources define the sound-engine side of the research: what FM is, where the DX lineage matters, and which iPad products already cover adjacent ground.

Expressive touch instruments

These sources keep the iPad surface from being treated as a mouse replacement. The serious precedent is playable touch, not better parameter editing.

Mapping, passive sensing, and new instruments

This cluster gives SensorSynthFM its design spine: the mapping is part of the instrument, and more inputs only help if the relationship stays learnable.

Performance workflow and sequencing

These sources keep the project honest about where iPad instruments actually live: hosts, loopers, routing tools, modular environments, and tempo-sync systems.

Technical stack and cognitive load

This cluster turns the research into build constraints: AUv3 discipline, latency awareness, and an interface that explains cause and effect quickly.

Inclusive Source Atlas for Review

This local draft puts the broad public-safe source inventory on the page first. James can read it by topic, then strike irrelevant items after seeing the full field instead of pre-filtering a spreadsheet.

The atlas links 319 public-safe URLs. Sources can appear in more than one topic because the same source can support FM, workflow, touch, and sensor arguments at once. 10 candidate URLs are held out by the safety gate because they need manual review, replacement, or removal before public use.

319public-safe linked URLs
575topic placements
1383full-corpus recovery entries behind the review

Internal corpus material is summarized by coverage count only. Local paths, private Drive material, raw run-state artifacts, and private notes are not exposed as public links.

FM synthesis and iPad instruments

111 sources

FM theory, DX lineage, iPad synth products, and adjacent instrument references that define the sound space.

Touchscreen expression and glass-interface constraints

87 sources

Playable touch surfaces, MPE, Apple Pencil, glass-instrument limits, and expressive iPad performance.

Sensor-driven and environmental modulation

65 sources

Sensor input, gesture mapping, environmental signal, and new-instrument research behind the room-shaped modulation idea.

Performance workflow, looping, and sequencing

81 sources

Host apps, loopers, routing, sequencing, tempo sync, and the actual workflow ecology around iPad instruments.

iPadOS audio, AudioKit, AUv3, MIDI, and latency

106 sources

Implementation-facing references for iOS audio, AudioKit, AUv3, MIDI, latency, and framework decisions.

Visual feedback and state legibility

39 sources

Sources about making instrument state visible, readable, and musically explainable.

Accessibility and cognitive accessibility

23 sources

Accessibility, cognitive load, inclusive-design, and learnability sources relevant to the thesis demo.

  • Accessible Digital Musical Instruments - A Review of Musical Interfaces in Inclusive Music Practicedoi.org
    accessibility and cognitive accessibility implicationsvisual feedback and state legibility
  • Accessible Digital Musical Instruments - A Review of Musical Interfaces in Inclusive Music Practicegenerativemusic.com
    accessibility and cognitive accessibility implicationsperformance workflow and sequencingvisual feedback and state legibility
  • Animoog Z - App Storeapps.apple.com
    FM on iPad and FM complexityaccessibility and cognitive accessibility implicationsiPadOS audio, AudioKit, AUv3, latency, MIDI, Ableton Link
  • Cook 2001 on Springerlink.springer.com
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Cook 2001, Principles for Designing Computer Music Controllers - NIMEnime.org
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Cook NIME 2007 keynote notesnyeaf07nime07.wordpress.com
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • GeoShred - App Storeapps.apple.com
    FM on iPad and FM complexityaccessibility and cognitive accessibility implicationsiPadOS audio, AudioKit, AUv3, latency, MIDI, Ableton Link
  • GeoShred Pro Review - AudioNewsRoomaudionewsroom.net
    accessibility and cognitive accessibility implicationsiPadOS audio, AudioKit, AUv3, latency, MIDI, Ableton Linkpassive/environmental sensing and sensor-to-FM mappings
  • GeoShred Studio desktop release - Synth Anatomy Oct 2024synthanatomy.com
    accessibility and cognitive accessibility implicationsiPadOS audio, AudioKit, AUv3, latency, MIDI, Ableton Linkpassive/environmental sensing and sensor-to-FM mappings
  • HALhal.science
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Jordà Google Scholarscholar.google.com
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Jordà, Digital Instruments and Players - NIME 2004nime.org
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Jordà, Sonigraphical Instruments - Springerlink.springer.com
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • MIDI and MPE questions - Loopy Pro forumforum.loopypro.com
    accessibility and cognitive accessibility implicationsiPadOS audio, AudioKit, AUv3, latency, MIDI, Ableton Linkpassive/environmental sensing and sensor-to-FM mappings
  • MPE support thread, Audiobus forumforum.audiob.us
    accessibility and cognitive accessibility implicationsiPadOS audio, AudioKit, AUv3, latency, MIDI, Ableton Linkpassive/environmental sensing and sensor-to-FM mappings
  • MPE Supported Apps - Loopy Pro forumforum.loopypro.com
    accessibility and cognitive accessibility implicationsiPadOS audio, AudioKit, AUv3, latency, MIDI, Ableton Linkpassive/environmental sensing and sensor-to-FM mappings
  • Orphion - App Storeapps.apple.com
    FM on iPad and FM complexityaccessibility and cognitive accessibility implicationsiPadOS audio, AudioKit, AUv3, latency, MIDI, Ableton Link
  • Orphion - iPadLoops reviewipadloops.com
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Orphion NIME 2014 papernime.org
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Rovan, Wanderley et al., Mapping Strategies (1997) - Semantic Scholarsemanticscholar.org
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Telemetron, Musical Instrument for Zero Gravity Performance, NIME 2018nime.org
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Wanderley Google Scholarscholar.google.com
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints
  • Wanderley, Gestural Control of Music - IRCAMrecherche.ircam.fr
    accessibility and cognitive accessibility implicationspassive/environmental sensing and sensor-to-FM mappingstouchscreen expression, MPE, Apple Pencil, and glass-instrument constraints

Additional project context and long-tail references

63 sources

Public sources that came through the larger corpus recovery and do not fit cleanly into one narrower topic bucket yet. They stay visible so James can strike or reclassify them during review.