First draft: A novel approach to generate sound using frequency modulation

WORK DOCUMENT!

Request for comments.

Project name: FMML (pronounced ‘fummel’)

Tags: resynthesis, spectral resynthesis, machine learning, frequency modulation synthesis, commercial applications, Yamaha FM, dexed.

Hypothesis: Is creating life-like or interesting sounds with FM sound generation with machine learning possible? For example, In a specific case, could we generate new old Yamaha DX7 (6 operators) or Yamaha TX81Z (4 operators) to be loaded with new patches created by this new way of sound generation?

Let us start with some basics. I have forgotten all my Fourier analysis, but a mix of sine waves could represent a sound. These sine waves must be modulated in amplitude over the course of the generated sound.

FM modulation is another beast. A modulator signal will change the frequency of the carrier signal.

Application and network architecture. It’s high level and not that correct at the moment. We need to look into suitable machine-learning configurations and frameworks.

We want to recreate a sound or have a sound that will inspire new sound creation for a Yamaha DX7— a pipeline on the top of my head.

  1. Load a sound PCM(?) -> convert format that is suitable to the ML input nodes.
  2. The network(The Creator) will parameterize a DX7 patch. The output nodes will deliver the patch parameters and we might need a parser that creates the specific SYSEX(sound patch file) that can program the synthesizer.
  3. The created patch will be fed into a simulated DX7 (dexed)
  4. Another network(The Evaluator) will compare the original sound with the generated sound and provide feedback to the Creator Network.
  5. What is to be satisfied? What is the product? The statistical network (The Creator network unattended by the Evaluator Network)? Is the training always needed?

Other thoughts.

Sysex streams and modulation from another device via MIDI to the DX7 — integration with, for example, an Arduino/Raspberry Pi.

Links:

https://mmontag.github.io/dx7-synth-js/

SYSEX file information.

The loaded patch file format is System Exclusive (SYSEX).

A Yamaha DX7 SYSEX (System Exclusive) file is a binary file format used to store the settings and parameters of a Yamaha DX7 synthesizer. The structure of a DX7 SYSEX file typically includes the following components:

  • Header: The header of the SYSEX file contains information about the file format, version, and possibly other metadata related to the data stored within the file. This header helps software or hardware devices recognize and interpret the SYSEX data.
  • Parameter Data: The bulk of the SYSEX file consists of parameter data representing the settings and parameters of the Yamaha DX7 synthesizer. These parameters include oscillator settings, envelope settings, modulation settings, and other parameters that define the sound and behavior of the synthesizer.
  • Checksum: A checksum is often included at the end of the SYSEX file to ensure data integrity during transmission or storage. The checksum is calculated based on the parameter data and is used to verify that the data has not been corrupted.
  • End of File Marker: Some SYSEX files may include an end-of-file marker or delimiter to indicate the end of the SYSEX data

SYSEX files can be transmitted between compatible MIDI devices or stored on a computer for later use. They allow users to save and transfer patches, presets, and other settings between different Yamaha DX7 synthesizers or software emulations. The specific structure and format of a SYSEX file may vary depending on factors such as the software or hardware used to create or interpret the file.

Word List:

MIDI

SYSEX

DEXED

FM