← SensorSynthFMSensorSynthFM is an iPad-native FM instrument shaped by touch, device motion, microphone energy, and environmental signals. The current checkpoint is direct note entry on the tabletop iPad, while the existing environmental modulation work remains available. The build runs on hardware and has a specific, documented list of what it still gets wrong. This page is the synopsis. The dated history, with every failure in order, lives in the full build log linked at the bottom.
A note on process. The code is written with AI assistance. I set the product direction, made the musical and interface judgments, ran every physical test described here, and I own every claim on this page.
What Works on the iPad
The accepted July 22 build passed 22 automated Xcode tests and a generic iOS build, then signed, installed, launched, and ran on my 11-inch iPad Pro. When the seven-day development signing profile later expired, a fresh Xcode run signed and installed the same build and it launched again. That interruption was a signing failure, not a product-code failure, and it is worth naming because build history should distinguish the two.
On August 3 I ran an eight-step cognitive walkthrough on the physical iPad, testing the instrument the way a first-time user would encounter it. Seven steps passed:
- Play was visible.
- Sound started, and the playing state made sense.
- The active source and FM destination were clear.
- Manual control and sensor modulation were distinguishable from each other.
- Microphone response was visible and audible.
- Setting a route to zero neutralized it, and restoring the amount resumed modulation.
- I could explain the premise of the instrument in one sentence.
The sensor mappings themselves earned their place through direct testing. Accelerometer magnitude routed to modulation index at +40 percent produced a clear change when the device moved; table taps and stationary vibration did not produce a useful result, which sets a real boundary on what the mapping is for. Gyroscope Y routed to modulator ratio at +41 percent changed timbre and apparent pitch predictably under slow tilting. Microphone amplitude routed to amplitude at +63 percent behaved the way an instrument source should: a quiet tone stayed stable, speech raised loudness proportionally, and claps produced louder transients with a slight apparent delay. That source is worth keeping if it is programmed and limited carefully.
There is also a handedness switch that moves auxiliary controls for left- and right-handed use. It works mechanically. It has not been tested with left-handed users, so I make no claim beyond the mechanism.
What the iPad Rejected
The eighth walkthrough step failed, and the failure is the most important finding in the current build. Accelerometer modulation changed the sound with no detectable visual response. Gyroscope response was visible but too subtle to trust. The screen did not adequately show what the sensors were doing to the sound. For an instrument whose whole premise is sensor modulation, that is not a cosmetic gap. It is the core promise going unexplained on screen.
Scrolling is permitted. Every eligible route still has to remain accessible with readable source and destination context and usable touch targets. The destination most recently edited should come into view without sensor updates stealing focus or moving controls beneath an active finger. That focus behavior is a direction to qualify, not a completed feature.
The walkthrough also exposed a discoverability failure in routing. Selecting a zero-valued route cell did not show the route as active. The route only became apparent after the amount moved away from zero. The interface did not explain how to create a route or how to remove one. I could operate it because I built it. A new user would have no such advantage.
What Changed Because It Failed
Every major decision in this build traces to a physical failure, not to a preference. Earlier rounds on hardware produced their own corrections. System chrome, meaning the iPad status area and gesture regions that surround the application, covered controls in an early layout. The first route editor collided with the iPadOS bottom gesture zone. Microphone behavior was too subtle to trust. Fixed routing hid the range of the instrument. Sensor writes made manual controls look broken. A side panel made the primary amount control too narrow, and the native layout hid the ninth source entirely. Each of those failures forced a design change that survives in the current build.
The August 3 walkthrough adds a new required list:
- Selected zero routes stay visibly selected.
- The interface explains that moving the amount activates a route.
- Every active route gets a visible X or remove control. Double-tap may remain a shortcut, but it cannot be the only removal method.
- Sensor response is shown clearly, so the ear and the eye agree.
- Scrolling is permitted when needed, but every eligible route must remain reachable with readable context and usable touch targets.
What does not change: familiar synthesis controls, bipolar amounts, a visible zero state, large touch targets, and clear source-to-destination routes. Those are the parts the walkthrough validated, and they hold.
Requirements Under Test
SensorSynthFM is now built and judged under a managed requirements method. The method comes from Jeremy Dick, Elizabeth Hull, and Ken Jackson, Requirements Engineering, Fourth Edition. It separates stakeholder outcomes in the problem domain from system behavior in the solution domain, and it gives every managed requirement a source, a rationale, requirement attributes, a qualification method, and traceability through implementation and evidence. The organizing question is not whether the feature was built. It is what would convince the stakeholder that the requirement has been satisfied. A passed implementation test does not by itself qualify a stakeholder outcome, so passing code and satisfied requirements remain separate claims. Official book page.
What follows is a readable public projection of a fuller requirements record. Stable requirement IDs now travel through the working task board and the dated public history, so a decision made today can be traced back to the requirement it answers. The current evidence comes from one expert maker on one physical 11-inch iPad Pro, including the August 3 eight-step walkthrough in which seven steps passed. It is not outside-user validation.
Open the full SensorSynthFM requirements board in FigJam. It carries the complete requirements map and the latest design-iteration annotation. This page remains the readable build summary.
Every enabled sensor route must make source, amount, destination, live response, and neutral zero state audible and visible.
- Why it exists
- Sensor modulation is the instrument's central promise. Audible change without visible cause reads as random or broken.
- Qualification
- On the physical iPad, isolate a route, create a source change, and identify source, destination, direction, and visible response before explanation.
- Current evidence
- Microphone response passed; accelerometer response was not visibly detectable; gyroscope response was too subtle to trust.
A selected route at zero must remain visibly selected and explain that changing the amount activates modulation.
- Why it exists
- Zero is an editing state, not the absence of a selection.
- Qualification
- Starting at zero, identify the selected route and the activation action without prompting.
- Current evidence
- The selected zero route did not look active, and the interface did not explain route activation.
Every active route must provide a visible, accessibly named removal control and a straightforward recovery path.
- Why it exists
- A shortcut gesture may help, but it cannot be the only way to remove a route.
- Qualification
- Remove and restore a route without being told a hidden gesture.
- Current evidence
- The removal control and recovery path are specified, but they have not been qualified on the current build.
The landscape modulation surface must keep all nine current sources and eligible routes accessible, with readable source and destination context and route cells at or above the accepted 56-point touch baseline.
- Why it exists
- Scrolling may be necessary as the matrix grows, but it cannot hide context, steal focus, or make the instrument harder to touch.
- Qualification
- On the 11-inch iPad in landscape, use scrolling as needed to verify every eligible route, readable source and destination context, and usable route cells, then confirm the most recently edited destination comes into view without sensor updates moving controls beneath an active finger.
- Current evidence
- The August observation that the ninth source required horizontal scrolling remains historical evidence. Scrolling is permitted, but contextual focus behavior and complete touch-target qualification remain unverified.
Manual base values and live sensor contribution must remain distinct, and returning a route to zero must neutralize sensor contribution without destabilizing sound.
- Why it exists
- The musician needs control authority and recovery.
- Qualification
- Move a route through positive, zero, and negative amounts while comparing base and live values and checking sound stability.
- Current evidence
- Passed James's owner-operated walkthrough; no outside-user evidence exists.
Microphone modulation must remain visibly and audibly causal while quiet input stays stable and louder speech or transients increase response without runaway pumping for two minutes.
- Why it exists
- Environmental input should shape sound without taking over the instrument.
- Qualification
- Record quiet baseline, speech, and clap response on the physical iPad for two minutes, and judge causality, stability, pumping, delay, and musical usefulness.
- Current evidence
- Passed one maker-operated test; exact calibration and transient delay remain open.
Before note-entry code is accepted, the existing note-entry interaction contract must define pitch mapping, touch-to-sound timing, polyphony, multi-touch, pressure or area, glide, scale and range behavior, established handedness, feedback, accidental activation, recovery, accessibility, and explicit non-goals for the direct-playing-first checkpoint.
- Why it exists
- SensorSynthFM must become a playable instrument without turning a solution idea into an unexamined requirement.
- Qualification
- Review every named attribute, source, edge case, rejected alternative, and physical acceptance test before implementation.
- Current evidence
- The existing note-entry interaction contract and established handedness are the starting point for today's direct-playing-first checkpoint; note-entry behavior is not yet physically qualified.
Two of the seven are failed on the current build, three are accepted and unqualified, and two hold on evidence produced by one maker on one device. Those numbers are the honest state of the instrument today, and they stay on this page until the evidence changes them.
Design iteration · September 5, 2026
September 5: note entry first
A twenty-question, AI-assisted design interview returned the project to its missing core: playing notes directly on the tabletop iPad. The next native checkpoint is starting, holding, changing, and releasing notes reliably, then playing phrases and chords while the existing environmental modulation remains available. It is not the full capture, sequencing, or patch-generation system.
The handedness switch is not a new idea from this interview. Earlier requirements already specify a mirrored off-hand control area and distinguish control touches from note touches. The interview reaffirmed that design and the target of ten independently sounding notes. Actual stable capacity still has to be qualified on the iPad; an expectation that most playing uses fewer fingers is not user evidence or a lower product target.
Established Ableton interaction patterns remain the reference wherever they apply. For capture, that means retaining recent playing, using the existing musical context when there is one, and allowing the selected phrase and loop boundaries to be changed. Extending that behavior to continuous environmental signals still needs design and physical testing.
Longer-term direction is preserved: deliberate environmental patch seeds, musically constrained patch profiles, optional regeneration with parameter locks and Undo, live environmental modulation, and editable captured modulation. Fuller patch randomization remains an exploration preference, not a settled generation algorithm. These capabilities do not become prerequisites for the first playable checkpoint.
This interview records the owner's design intent and prioritization. It does not establish outside-user usability, ten-voice hardware performance, validated camera sensing, or advisor approval. The next evidence comes from playing the native surface on the physical iPad and observing where the interaction breaks.
Matrix correction: Scrolling is permitted. Every eligible route must remain accessible with readable source/destination context and usable touch targets. The destination most recently edited by the player should be brought into view without sensor updates stealing focus or moving controls beneath an active finger. That focus behavior is a direction to qualify, not a completed feature.
The Next Field Pass
The next outside research idea is a visit to an Apple Store to study how users naturally approach and use display iPads. The questions are basic and directly relevant to this instrument: portrait versus landscape, where the first touch lands, one-hand versus two-hand use, reach, grip changes, keyboard and stand use, and how users recover from confusion. An instrument that lives on a tablet should be shaped by how users actually hold tablets, not by how I hold mine.
The first visit is only to understand the setting. Before any formal observation, I will settle the research boundary and obtain any required store permission. I will not record names, demographics, photos, audio, video, or quotes, and I will not interrupt anyone. That is the limit of what is planned.
Current Boundary
Every result on this page comes from one user testing on one device. The walkthrough, the mapping passes, and the failures are all mine, on my iPad. That is enough to drive design, and it is not enough to validate it. The handedness switch is untested with left-handed users. The discoverability fixes are specified but unproven. The project still needs testing with users other than me, and until that happens, the claims here stay scoped to what I observed directly.
Links
- Full build history, the dated log of every build, failure, and correction
- Desk research, the background work on FM synthesis and touch instruments
- Interactive matrix layout study, the browser-based exploration of the matrix layout
- GitHub, the source repository
The next product checkpoint is direct native note entry: reliable note lifecycle, phrases and chords, handedness-aware side controls, and measured capacity on the physical iPad. Existing bounded repair and device gates remain valid, but further matrix expansion and parameter-following focus are not prerequisites. Consequential work keeps its requirement ID, qualification test, and evidence state as it moves through implementation and the dated history.