Kent State MS UX CapstoneiPad FM InstrumentPlayable Note EntryBuild of November 15, 2026
Recorded on the iPad · September 20, 2026
The instrument can be played.
Since the September 20 build, I can start, move, hold, merge, and selectively release notes directly on the iPad. The loop below was recorded on the iPad that day.
Three active touches become a held chord. The orange reticles remain visible around stationary fingers; green edge markers show notes that continue after release. Screen recording from the iPad.Text description
Three touch points appear on the iPad surface. Orange reticles mark the active fingers, while green edge markers show held notes continuing after release.
Research and process
Build history and sources
The build history lists each build by date, including the failures. The desk research lists the sources.
Recruitment is underway for early, moderated usability sessions with the working iPad prototype. I built a short questionnaire in Kent State’s Qualtrics to understand interest, describe each volunteer’s background, and arrange follow-up.
What the screener asks
Eligibility and interest: adult status and interest in trying an unfamiliar iPad musical instrument.
Background: music-making activities, optional recent synthesizer use, and prior exposure to the project.
Practical arrangements: possible session areas, availability, and contact permission.
Recording choices: willingness to be recorded, with separate choices about public excerpts and keeping the private recording.
Previous music-making or synthesizer experience is not required. Beginners are welcome. I review responses individually and arrange sessions directly; completing the screener does not book a session.
Audio and video recording is required for participation. Permission to publish excerpts and permission to retain a private recording are separate choices, confirmed through session consent.
The sessions will examine how participants approach the instrument and where they encounter confusion. Findings and resulting design changes will be added after the sessions.
Demonstration only. The mockup does not collect responses. The live recruitment questionnaire is distributed separately.
Actual Qualtrics preview
Desktop and mobile views of the published participant questionnaire, captured in Qualtrics preview. These images document the form and its questions, not participant responses or study findings.
Any visible selections are demonstration inputs used to navigate the preview.
A 45-year-old man who produces techno, has a handful of releases, occasionally plays shows, and is interested in technology. He has access to an iPad and takes music seriously, but earns no meaningful income from it.
His practice is semi-professional in experience and activity. He has practical music-making experience but need not be an FM-synthesis expert. No particular iPad model, DAW, occupation, or hardware collection is assumed.
What he wants from the instrument
Find distinctive sounds and turn experimentation into material for his own music.
Begin playing quickly without extensive setup or a theory lesson.
Explore touch and environmental influence while retaining intentional control of the sound.
Understand what caused a change and recover a useful musical state.
Find enough expressive depth to return after the novelty wears off.
How this guides the design
This persona keeps quick note entry, understandable sound changes, and recoverable control central to the design. SensorSynthFM adds another expressive instrument to his practice; it does not replace a complete production setup.
Basis and limits
I selected this working persona after reviewing the customer-market research. Age 45 is illustrative. The techno, release, performance, and technology traits define the chosen target; their combination has not been measured in the audience.
The goals above are design hypotheses to check in participant sessions. The recruiting pool is broader than this persona, including complete beginners. Volunteers do not need to match his age, gender, genre, release history, or equipment.
AI assistance supported research synthesis and drafting. I selected the persona and recruiting approach.
The Idea
Most synth interfaces ask the player to manage a dense parameter editor. SensorSynthFM starts from a narrower bet: an iPad can become a small FM instrument that listens to touch, motion, surface vibration, and room energy, then turns those signals into timbre.
That only works if the player can see what each sensor is doing. Without that, the instrument feels random.
The main screen is for playing notes. The modulation matrix, the grid that routes each sensor to a sound parameter, remains available. Touch enters notes in Quantized or Freehand mode. The sensors shape the tone but do not take over note or pitch control.
The Thesis
Sensor modulation gave the project a point of view, but the prototype could not be played directly. The September 20 build puts note entry at the center: touch starts a voice, vertical movement changes pitch, multitouch makes chords, and Hold lets those chords remain editable after the fingers leave the glass.
Playing on the iPad changed the feedback design. A 38-point touch halo disappeared beneath a fingertip.
The next question is whether another musician can understand it without my help.
What Exists
FM engine
The build uses a fixed bank of ten voices built on AudioKit, an open-source audio library. Each touch owns its pitch while global FM controls and sensor input continue to shape timbre. Each note has its own envelope and filter, and all notes share echo and reverb. On September 7 I held five notes steadily and registered ten touches at once.
Note entry and Hold
Quantized and Freehand modes share the same landscape surface. Active reticles follow the finger, labels show note and frequency, Freehand adds signed cents, and fixed edge markers show held notes after the finger lifts. With Hold on, I can retrigger a note, slide it to a new pitch, merge it into a pitch that is already held, remove one note, or release them all.
Live sensor mapping
The matrix routes ten sources to nine destinations. In the starting sound, device motion and left-right finger position drive modulation index, finger position also opens the filter, microphone level drives loudness, and tilt drives modulator ratio. Every route can be changed or cleared on the Matrix page.
Instrument page
Sliders set the fade in and fade out of each note, a low-pass filter with resonance, and the amount of echo and reverb. RESET START restores the starting sound, and RELEASE ALL stops every note.
Scene description
A scene-description layer is written and under review. It turns raw sensor signals into descriptors such as room energy, surface impact, vibration, brightness, and bass pressure. Xcode tests and a generic iOS build passed on July 13. I have not yet tested on the iPad whether those descriptors are musically useful.
What Changed After Testing on the iPad
Listening on the iPad, I ranked the gyroscope first and the accelerometer second. The microphone response was too subtle to trust until I added diagnostics. After that it was stable and clearly tied to the sound.
A touch-first matrix replaced the hard-coded routing with nine sources, four targets, and 36 route cells, each adjustable in either direction. I signed, installed, and launched it on the iPad. The first route editor sat too close to the iPadOS gesture zone at the bottom of the screen and failed. After the September 6 repair, I could turn a route on, remove it, and restore it.
The September 20 build passed 57 Xcode tests after two failing tests were fixed. I then checked Hold and the touch feedback on the iPad.
In November, moving four or five fingers at once cut off every note. iPadOS takes that motion for a system gesture. Guided Access, which locks the iPad to one app, prevents it, and the sessions will run that way. Resonance on the first low-pass filter had no audible effect, so a ladder filter, a design with a stronger resonant peak, replaced it on November 15.
Documentation and Research
I started a project wiki because the code, research, and design notes had drifted apart.
The code lives on GitHub. The public repository trails the build on the iPad. The related Ableton evaluation suite examines expert music software.
How It Is Being Built
I do not have a traditional software development background. The build uses AI collaborators for implementation, review, and research synthesis while I make the musical and design decisions and do the testing.
Clarence, my AI system, runs that collaboration. It keeps the task board, research notes, and source lists in one place.
What Is Still Unresolved
As of November 15, 2026
The build reaches my iPad through TestFlight, the Apple service for distributing test builds. It is not available to the public. Still untested: ten notes sounding at once with the sensors running, and left-handed play by a left-handed player. Automatic scrolling to the destination being edited is not built.
Next
Next come five to seven moderated sessions with consenting people who did not design the interface, including a left-handed participant where possible. Recruitment is underway and no session has run. Chord-name feedback for learners is a possible later requirement. The looper, sequence editor, patch generation, presets, MPE, AUv3, and Ableton integration are not part of the capstone. What happens in the sessions will guide what comes after.