This is the dated build history for SensorSynthFM, an FM synthesizer driven by an iPad's onboard sensors. It replaces a single stale July 1 process snapshot with the full record from inception forward. I keep it because the project's hardest problem was never a shortage of ideas. It was deciding which ideas survived source checks, tests, and physical use.
The record combines public Git history, research and design notes, physical-device reports, publication receipts, and a reviewed projection of the working Kanban board. AI collaborators help with implementation, review, and research synthesis. I own the musical judgment, UX direction, hardware testing, evidence gates, and public claims. That boundary stays visible.
The current build has crossed several physical gates. It has not crossed all of them. These are three different states, and treating them as one would turn the page into a sales document.
Physical evidence
Audio output, accelerometer and gyroscope modulation, microphone response, a causal two-minute microphone route, modulation-matrix launch, and the revised route controls all reached physical hardware. Exact commit 635ab09 passed 20 Xcode tests, a generic iOS build, physical signing, installation, and launch. James then confirmed that the default minus-then-plus order, removal of the dedicated ZERO control, and signed-amount reset all worked on the iPad.
Physical interaction
The quick route-control repair passed physical use. The native screen still differs from the public layout study, and that full composition has not been implemented in Swift. One physical screenshot, the native-versus-study delta, matrix-growth behavior, center haptics, handedness, and bottom-gesture avoidance remain open. The accepted controls do not settle the entire modulation page.
Scope boundary
The first Performance-page note-input definition is active design work. The looper, sequence editor, four-operator engine, presets, MPE, AUv3, Ableton integration, and direct gesture editing inside matrix cells are not current implementation claims. The design can move forward without pretending those systems exist.
Design note · July 17, 2026
Nine sources by four targets is the current proof, not the final instrument. The matrix has to absorb more synthesis structure as the engine grows: additional operators, envelopes, LFOs, macros, effects, and destinations. It cannot do that by shrinking touch targets until the interface becomes a desktop routing table under glass.
This study turns the July 15 physical feedback into one reviewable composition. All nine current sources stay visible. The selected route sits directly below the matrix. The nudge order becomes minus then plus. The signed amount becomes the reset target, and the right rail carries secondary context. This is the direction under review. It is not a finished matrix or a claim that the current synthesis engine already needs every future lane.
Interactive design study. Select routes, drag the bipolar amount, use the one-percent nudges, reset through the signed value, switch handedness, and compare the current composition with the revision. This changes the design record, not the SensorSynthFM application.
Open the full interactive study ↗The screenshots are generated from the working board, then reduced to approved public milestone titles, status, and abstracted work lanes. The current source board holds 2 ready, 3 todo, 2 needs-decision, 45 done, and 1 scheduled cards. The image does not expose private card bodies, comments, paths, logs, or raw identities.
First public board state
Open full-size screenshot ↗Layout-study milestone board
Open full-size screenshot ↗Current milestone board
Open full-size screenshot ↗This chronology records product behavior, design decisions, research gates, publications, failures, and open validation. It does not inflate internal bookkeeping into progress.
FoundationVerified
SensorSynthFM started as an Xcode project, not a product claim. The first commit established the application shell and a source of truth that later design work could be checked against.
InteractionVerified
SwiftUI mockups and supporting project documentation gave the instrument its first visible interaction model. This was still interface intent. No hardware evidence existed yet.
SensorsVerified
The README established the working project description. SensorManager added microphone permission handling and corrected an FFT memory leak, which turned passive sensing from a diagram into a runnable code path.
AudioVerified
A sine-wave test proved that the audio engine could run. It was deliberately plain. The point was to establish output before adding FM complexity.
DesignDecision
Project notes and design documents were expanded before the synthesis engine changed. The work started to name the intended behaviors, not only the files needed to produce them.
AudioVerified
The sine wave was replaced by AudioKit's FMOscillator. Carrier frequency, modulation index, modulator ratio, and amplitude became the first real synthesis controls.
IntegrationVerified
SensorFMBridge and SensorFMTestView connected accelerometer magnitude, microphone amplitude, and gyroscope Y to FM parameters. This created the first integrated proof surface for judging whether a physical signal caused an audible change.
DocumentationPublished
Public project documentation was added and a project Wiki was initialized. The code, README, design documents, and longer plans had begun describing different versions of the instrument. That was a truth problem.
AuditDecision
The canonical repository and a duplicate starter repository were re-audited. The next gate became physical activation of the existing integrated view, not another feature branch.
InteractionDecision
Touch-native looping, sequence and edit grammar, rhythmic capture, and a three-page architecture were documented. They stayed behind the activation gate. The design could grow after the physical proof, not instead of it.
RestartDecision
The restart plan narrowed the work to physical activation. GitHub and the canonical repository remained code authority. Google Drive remained an archive and handoff surface, not a place to run the project.
CapstoneVerified
Capstone coursework was mapped to the active build, the research corpus was recovered, and the desk-research inventory was prepared. The long-running Kanban structure was prepared as the operational evidence trail for research, implementation, review, and parked scope.
HardwareVerified
The app entry was changed to SensorFMTestView. James ran the integrated build on a physical iPad and confirmed sound output plus audible accelerometer and gyroscope mappings. The microphone seemed active, but the response was too subtle to trust.
PublicationPublished
The desk-research page and first process page were published with a 33-card Kanban capture. Capstone evidence, discovery questions, research recovery, parked ambitions, and the usability-video privacy boundary became explicit parts of the build record.
ResearchPublished
Repository documentation drift was corrected and the SceneFingerprint implementation packet was clarified. An inclusive source atlas built from 319 reviewed public-safe URLs was published so the research surface could be inspected before it was narrowed.
EvaluationDecision
Physical listening ranked the gyroscope first and the accelerometer second. The microphone was alive, but it needed a different interpretation. Accessibility requirements, an interaction model, a cognitive walkthrough, a moderated usability script, and findings-to-decisions traceability were documented around that evidence.
ImplementationOpen
Microphone diagnostics, safe-area and control-authority repairs, and SceneFingerprint implementation moved into review lanes. The public project narrative was corrected at the same time so the page stopped promising a fixed routing model as the finished direction.
ScopeDecision
The current focus narrowed to one convincing two-minute environmental-modulation demo with the iPad resting on a table. The project stopped treating whole-device motion as the main performance gesture. This is room and surface modulation, not keytar behavior.
DiagnosticsVerified
Microphone diagnostics and a SceneFingerprint proof harness were assembled on a review branch. Xcode tests and a generic iOS build passed. James cleared the signing boundary and reported that the microphone responded on the physical iPad.
InteractionVerified
A touch-first modulation matrix was implemented with nine sources, four targets, 36 bipolar route cells, editable base values, and migrated defaults. Seventeen Xcode tests and the generic iOS build passed. James reported physical signing, installation, and launch success, while per-route interaction behavior remained under review.
HardwareVerified
James ran a two-minute stationary microphone test in a room measuring roughly 60 dB. The route was audibly causal, stable, and only slightly pumping, although exact calibration remained open. The same physical pass exposed a blocking defect: the route slider occupied the iPadOS bottom gesture zone.
ImplementationVerified
A local adaptive route inspector defaulted to the right, mirrored for left-handed use, moved diagnostics behind progressive disclosure, and added a center-zero control with one-percent nudges and a zero detent. Source checks passed. The next physical review rejected its fixed side rail, turning this implementation into evidence for the redesign rather than a finished interaction.
HardwareOpen
The repaired build reached the physical 11-inch iPad with the selected route fixed at the right edge. That solved the wrong problem. The 300-point rail made the primary bipolar slider too narrow for confident touch control and left most of the right column unused. The next layout must put a wider slider directly below the matrix and reserve the right rail for secondary elements.
InteractionDecision
The selected-route editor moved below the matrix. Twenty Xcode tests, a generic iOS build, and simulator geometry checks passed before James signed, installed, and launched exact commit c5d3472 on the physical 11-inch iPad without errors. The physical review was clear: the revision is better, but it still underuses the available space. The dedicated ZERO control wastes room, the plus/minus nudge order must reverse, and the grid should move toward an Ableton-like layout over time. The next interaction revision remains open.
EvidenceDecision
A live check confirmed that Xcode 26.3 still sees the physical iPad as paired. The working capture path is Window > Devices and Simulators, where a human takes the full-resolution screenshot before the existing transfer, sanitization, privacy review, and publication pipeline runs. Apple's newer Device Hub remote-control instructions do not apply to the installed toolchain. Agent-operated physical UI-test capture remains proposed, not implemented.
DesignPublished
The physical rejection became an interactive current-versus-revision design study. The revision keeps all nine current sources visible, gives the selected route a wide bipolar strip below the grid, reverses the nudge order, turns the signed amount into the reset target, and reserves the right rail for secondary context. The harder constraint is growth: nine sources by four targets is the current proof, not the final ceiling. The matrix has to expand with the synth without collapsing touch targets or burying route editing.
HardwareVerified
Exact commit 635ab09 removed the dedicated ZERO control, reversed the default nudge order, and made the signed amount the reset target. Twenty Xcode tests, a generic iOS build, installation, and launch passed before James confirmed all three requested controls worked on the physical iPad. The larger web-study composition is still not in Swift. The new board keeps that decision open while making full-chain integration, note-entry definition, walkthrough, mapping survival, and the capstone evidence brief visible in sequence.
This page is event-driven. It is not a real-time board dump and it does not update on a timer. A timer can copy a status. It cannot decide whether a code change earned a public claim.
Every verified public milestone now closes through the same history gate. James's July 15 request is standing approval for narrow, public-safe history refreshes that pass this sequence. It is not approval to publish unrelated code, private research, participant material, or unsupported claims.
Step 1
Start with the strongest evidence available: Git, Xcode output, a physical-device report, a research artifact, or a live publication receipt.
Step 2
State what passed, what failed, and what remains unverified. Classify each new board title before any part of it can reach the public screenshot.
Step 3
Produce the current board image and a date-keyed archive, then run the privacy guard, typecheck, lint, and production build. A timer cannot do this judgment.
Step 4
Verify the source workflow, public Pages workflow, live route, and image URLs. Internal smokes, retries, bookkeeping, and unverified code churn do not earn a history entry.
I will keep documenting what survives physical use, what fails under a hand, and what stays open after the code looks finished. The device gets the last word.