#4624 Sheet Music to Knowledge Graph (#2)

#4624 When applying sheet music to a knowledge graph, what level (notes, stanzas, page, etc.) would it make sense to create nodes for in the graph?

When modeling sheet music in a knowledge graph, the optimal node level depends heavily on your primary objective: analytical query engine (harmonic analysis, pattern searching, voice leading) versus layout/rendering engine (MusicXML, digital page display).

A robust, flexible schema typically follows a multi-tier hierarchical tree supplemented by horizontal sequence relationships (NEXT, SIMULTANEOUS_WITH).

1. Core Hierarchical Nodes (Macro to Micro)

Tier 1: Metadata & Form

Work / Composition: The top-level entity representing the piece as an abstract concept.
- Properties: Title, Composer, Catalog Number (Opus), Key Signature, Genre, Year.
Movement / Section: Major structural divisions within a piece.
- Properties: Section Name (e.g., Exposition, Chorus, Allegro), Form, Tempo Marking.

Tier 2: Structural & Temporal Framework

Measure (Bar): The primary anchor for temporal alignment across multiple instruments or voices.
- Properties: Measure Number, Time Signature, Key Signature, Repeat Markings.
- Why a node? Measures allow you to align different instruments vertically at a specific point in musical time.

Tier 3: Voices & Performing Units

Part / Voice: Defines the instrument or vocal range performing the notes.
- Properties: Instrument Name, Transposition, Clef.

Tier 4: Musical Events (The Core Atomic Level)

Event / Chord: Represents a specific temporal slice or beat within a measure.
- Properties: Offset / Beat Number, Duration, Dynamics ( $p$ , $f$ , $crescendo$ ).
Note / Rest: The individual pitch component.
- Properties: Pitch Name ( $C_4$ ), MIDI Pitch Number (60), Frequency (Hz), Articulation (staccato, accent), Duration.

2. Should `Page` and `System` Be Nodes?

Physical score layout elements (Page, System, Staff) serve a fundamentally different purpose than musical data:

Node Type	Keep as a Node If...	Store as a Property/Metadata If...
`Page` / `System`	You are building an OMR (Optical Music Recognition) pipeline, page-syncing audio player, or layout renderer where physical coordinates matter.	You are performing musical analysis (harmonies, melodies, motif searching). Page breaks are arbitrary visual artifacts that break musical continuity.
Note	You want to trace melodic pathways, compute interval jumps, or do graph algorithmic pattern matching across pieces.	You only care about macro-level metadata or metadata searching (e.g., "Find all pieces in G major with 3/4 time").

3. Structural & Sequence Relationships

The real power of a sheet music knowledge graph lies in its directional edges, allowing both horizontal (melodic) and vertical (harmonic) traversal.

(Work) ──[:HAS_SECTION]──> (Section) ──[:HAS_MEASURE]──> (Measure)
                                                            │
                                                     [:CONTAINS_EVENT]
                                                            │
                                                            ▼
                                                        (Event)
                                                       /       \
                                           [:HAS_NOTE]          [:HAS_NOTE]
                                               /                  \
                                        (Note: C4) ──[:HARMONY]──> (Note: E4)

Critical Edge Types:

Temporal / Horizontal Relationships:
- (n1:Note)-[:NEXT]->(n2:Note) or (m1:Measure)-[:NEXT_MEASURE]->(m2:Measure)
- Enables: Querying for specific melodic contours (e.g., find a leap of a minor 7th followed by step-wise motion).
Harmonic / Vertical Relationships:
- (n1:Note)-[:SIMULTANEOUS_WITH]->(n2:Note) or (e:Event)-[:HAS_CHORD_TYPE]->(c:Chord)
- Enables: Polyphonic analysis, identifying voice-leading parallelisms (e.g., parallel fifths), or analyzing harmonic progressions.
Structural Relationships:
- (p:Part)-[:PERFORMS]->(e:Event)
- Enables: Isolating a single instrument's path through the graph without losing its temporal link to other parts.

Recommended Default Strategy

If your goal is algorithmic music analysis, motif discovery, or MIDI/audio processing, model down to the Note/Rest level using Measures and Events as temporal anchors, while keeping Page/System layout metadata strictly as properties on the Measure or Note nodes.

What specific type of analysis or application are you building this knowledge graph for?

When looking for recurring patterns within music, what are the smallest unit of music that would make sense in a knowledge graph

When looking for recurring patterns—whether melodic motifs, harmonic progressions, rhythmic cells, or textural gestures—the absolute smallest unit that makes sense to represent as a discrete node in a knowledge graph is the Note Event (or Pitch-Duration Event).

However, if you only store raw Note nodes, querying for abstract or transposed patterns across a graph can become computationally heavy. To capture recurring patterns efficiently, music graph schemas typically use three fundamental micro-units:

1. The Atomic Unit: `Note Event`

At the lowest physical layer, a Note Event node captures the concrete realization of a sound in time.

(:Note {pitch: "C4", midi: 60, duration_ticks: 480, onset_time: 0.0})

Why it’s the minimum: You cannot reliably decompose music further without losing musical meaning (e.g., decomposing a note into audio samples or physical frequencies yields digital signal processing data, not symbolic music data).
When to stop here: Restricting your graph to Note Event nodes works well if you only care about exact pitch/rhythmic matches in a specific key and octave.

2. The Abstraction Unit: `Interval` (The Pattern Engine)

If your goal is finding recurring patterns (like a recurring 3-note leitmotif or a specific rhythmic hook), pitch names ( $C_4 \rightarrow E_4 \rightarrow G_4$ ) are often too rigid because motifs frequently recur transposed into different keys, inversions, or octaves.

To solve this, model the Interval (or Transition) as a first-class node or explicit edge between sequential Note Events.

(Note: C4) ──[:TRANSITION {semitones: +4, duration_ratio: 1.0}]──> (Note: E4)

Interval Vector / Relative Motion: Storing melodic interval distance (in semitones or diatonic steps) and temporal ratios ( $\Delta t$ ) allows you to query for identical contours across entirely different keys or tempos using simple Cypher pattern matching:
$\text{Pattern} = (+4 \text{ semitones}) \xrightarrow{} (+3 \text{ semitones})$
$\text{Pattern} = (+4 \text{ semitones}) \xrightarrow{} (+3 \text{ semitones})$
Why this matters for graphs: It turns a pattern-matching query from "Find all notes $C_4 \rightarrow E_4 \rightarrow G_4$ " into "Find all subgraphs with edge chain $+4 \rightarrow +3$ ."

3. The Functional Micro-Unit: `Pitch Class` & `Degree`

To catch recurring harmonic or scalar patterns regardless of octave or arrangement, link your Note Event nodes to static canonical PitchClass or ScaleDegree nodes.

(Note: C4) ──[:INSTANCE_OF]──> (PitchClass: C)
(Note: C4) ──[:FUNCTION_AS {in_key: "C_Major"}]──> (ScaleDegree: 1)

Pitch Class ( $0–11$ or $C–B$ ): Collapses all octaves ( $C_1, C_2, C_3\dots$ ) into a single structural node.
Scale Degree ( $\hat{1}, \hat{2}, \hat{3}\dots$ ): Allows you to search for functional harmonic patterns (e.g., finding every instance of a $\hat{5} \rightarrow \hat{1}$ cadence, regardless of key).

Summary of the Micro-Hierarchy

To optimize for pattern matching, the standard graph representation at the lowest level looks like this:

Micro-Unit Node	Operational Level	Pattern-Matching Role
Note Event	Atomic Concrete	Identifies specific, absolute occurrences in a score.
Interval / Transition	Relational / Dynamic	Detects transposed melodies, contour shapes, and rhythmic motifs.
Pitch Class / Scale Degree	Harmonic / Abstract	Detects key-agnostic chord progressions, scale movements, and voice leading.

If you model down to Note Event nodes connected via directed Interval relationships, you give the graph engine the exact mathematical granularity needed to match both exact notes and abstract thematic motifs.

Describe an Python application to read sheet music and add both #2 recurring patterns and #3 recurring harmonic or scalar patterns to the knowledge graph, with associations preceded-by and followed-by

Here is a blueprint and reference implementation for a Python application that reads symbolic sheet music (via music21), extracts melodic and harmonic patterns, and ingests them into a Neo4j knowledge graph using directional PRECEDED_BY and FOLLOWED_BY relationships.

1. System Architecture & Data Flow

[ Sheet Music (MusicXML/MIDI) ]
                               │
                              ▼ (music21 Parser)
   ┌──────────────────────┐
   │ Temporal Event Slice                      │
   └──────────┬───────────┘
                                │
      ┌─────────┴────────────┐
      ▼                                                      ▼
[ Melodic Stream ]                      [ Harmonic Stream ]
 (Note-to-Note)                              (Chord / Vertical)
      │                                                        │
      ▼                                                      ▼
 Relative Intervals                        Pitch Classes / Scale Degrees
      │                                                         │
      └───────┬──────────────┘
                           │
                          ▼ (neo4j Python Driver)
[ Neo4j Graph Database ]
  (NoteEvent) ──[:PRECEDED_BY | :FOLLOWED_BY]──> (NoteEvent)
  (NoteEvent) ──[:HAS_INTERVAL]──> (Interval)
  (NoteEvent) ──[:HAS_DEGREE]────> (ScaleDegree)

2. Graph Schema Design

To ensure optimal performance for motif and harmonic pattern queries, we model both explicit events and their relational abstractions:

Nodes

:NoteEvent: Unique instance of a pitch occurrence in the score (id, pitch_name, midi, onset_beat, duration).
:Interval: Abstract relative transition (semitones, diatonic_steps, duration_ratio).
:ScaleDegree: Functional harmonic unit (degree, key_context).
:PitchClass: Pitch representation independent of octave (value 0–11, e.g., $0 = C$ ).

Relationships

(:NoteEvent)-[:FOLLOWED_BY]->(:NoteEvent): Sequential time flow (forward).
(:NoteEvent)-[:PRECEDED_BY]->(:NoteEvent): Sequential time flow (backward traversal optimization).
(:NoteEvent)-[:HAS_INTERVAL]->(:Interval): Relational melodic motif connection.
(:NoteEvent)-[:HAS_DEGREE]->(:ScaleDegree): Functional harmonic context.
(:NoteEvent)-[:HAS_PITCH_CLASS]->(:PitchClass): Octave-agnostic pitch class association.

3. Python Implementation

Requirements

pip install music21 neo4j

Script: `music_graph_ingest.py`

import music21 as m21
from neo4j import GraphDatabase

class MusicGraphIngestor:
    def __init__(self, uri, auth):
        self.driver = GraphDatabase.driver(uri, auth=auth)

    def close(self):
        self.driver.close()

    def process_score(self, file_path, work_title):
        score = m21.converter.parse(file_path)
        
        # Analyze key for scale degree resolution
        key_context = score.analyze('key')
        print(f"Processing '{work_title}' in key: {key_context.tonic.name} {key_context.mode}")

        # Flatten score into a single timeline of note events
        # (Handling single-voice / primary melodic voice stream)
        flat_notes = list(score.flat.notes)

        events = []
        for idx, element in enumerate(flat_notes):
            if isinstance(element, m21.note.Note):
                note_id = f"{work_title}_n_{idx}"
                pitch_class = element.pitch.pitchClass
                midi_val = element.pitch.midi
                
                # Functional Scale Degree calculation
                scale_degree = key_context.getScaleDegreeFromPitch(element.pitch)
                degree_str = str(scale_degree) if scale_degree else "Unknown"

                events.append({
                    "id": note_id,
                    "pitch_name": element.pitch.nameWithOctave,
                    "midi": midi_val,
                    "pitch_class": pitch_class,
                    "scale_degree": degree_str,
                    "onset": float(element.offset),
                    "duration": float(element.quarterLength)
                })

        # Calculate relative intervals between sequential notes
        transitions = []
        for i in range(len(events) - 1):
            curr_e = events[i]
            next_e = events[i+1]

            semitone_diff = next_e["midi"] - curr_e["midi"]
            dur_ratio = round(next_e["duration"] / (curr_e["duration"] if curr_e["duration"] > 0 else 1.0), 3)

            transitions.append({
                "from_id": curr_e["id"],
                "to_id": next_e["id"],
                "semitones": semitone_diff,
                "dur_ratio": dur_ratio,
                "interval_id": f"int_{semitone_diff}_r{dur_ratio}"
            })

        self._write_to_neo4j(work_title, key_context.name, events, transitions)

    def _write_to_neo4j(self, work_title, key_name, events, transitions):
        query = """
        UNWIND $events AS ev
        MERGE (n:NoteEvent {id: ev.id})
        SET n.pitch_name = ev.pitch_name,
            n.midi = ev.midi,
            n.onset = ev.onset,
            n.duration = ev.duration,
            n.work = $work_title

        // Abstract #3: Pitch Class
        MERGE (pc:PitchClass {value: ev.pitch_class})
        MERGE (n)-[:HAS_PITCH_CLASS]->(pc)

        // Abstract #3: Scale Degree / Functional Harmony
        MERGE (sd:ScaleDegree {degree: ev.scale_degree, key: $key_name})
        MERGE (n)-[:HAS_DEGREE]->(sd)

        WITH n, ev
        """

        rel_query = """
        UNWIND $transitions AS tr
        MATCH (a:NoteEvent {id: tr.from_id})
        MATCH (b:NoteEvent {id: tr.to_id})

        // Directional Sequence Edges
        MERGE (a)-[:FOLLOWED_BY]->(b)
        MERGE (b)-[:PRECEDED_BY]->(a)

        // Abstract #2: Relational Interval Nodes
        MERGE (inv:Interval {id: tr.interval_id})
        SET inv.semitones = tr.semitones,
            inv.duration_ratio = tr.dur_ratio

        MERGE (a)-[:HAS_INTERVAL_TO]->(inv)
        MERGE (inv)-[:LEADS_TO]->(b)
        """

        with self.driver.session() as session:
            session.run(query, events=events, work_title=work_title, key_name=key_name)
            session.run(rel_query, transitions=transitions)
            print(f"Successfully ingested {len(events)} events and {len(transitions)} transitions.")


if __name__ == "__main__":
    # Example Initialization
    # ingestor = MusicGraphIngestor("bolt://localhost:7687", ("neo4j", "password"))
    # ingestor.process_score("path/to/file.mid", "Composition_01")
    # ingestor.close()
    pass

4. Cypher Pattern-Matching Queries

Once the score is ingested, you can query both abstract melodic contours and functional harmonic progressions across keys.

Query A: Find Recurring Melodic Motifs (#2 - Transposed Interval Matching)

To find all occurrences of a specific 3-note melodic motif (e.g., a step up followed by a leap down regardless of starting key):

MATCH (n1:NoteEvent)-[:FOLLOWED_BY]->(n2:NoteEvent)-[:FOLLOWED_BY]->(n3:NoteEvent),
      (n1)-[:HAS_INTERVAL_TO]->(i1:Interval),
      (n2)-[:HAS_INTERVAL_TO]->(i2:Interval)
WHERE i1.semitones = 2 AND i2.semitones = -5
RETURN n1.work, n1.pitch_name AS start_note, n2.pitch_name, n3.pitch_name, n1.onset

Query B: Find Recurring Harmonic/Scalar Progressions (#3 - Functional Cadences)

To find every occurrence of a functional scalar sequence (e.g., $\hat{5} \rightarrow \hat{1}$ or authentic cadential motion) across different compositions:

MATCH (n1:NoteEvent)-[:FOLLOWED_BY]->(n2:NoteEvent),
      (n1)-[:HAS_DEGREE]->(sd1:ScaleDegree {degree: "5"}),
      (n2)-[:HAS_DEGREE]->(sd2:ScaleDegree {degree: "1"})
RETURN n1.work, n1.pitch_name, n2.pitch_name, sd1.key AS key_context

Query C: Backward Traversal using `PRECEDED_BY`

To trace backward from a specific pitch resolution or climax point to examine leading material:

MATCH (target:NoteEvent {pitch_name: "C5"})<-[:PRECEDED_BY*1..3]-(preceding:NoteEvent)
RETURN preceding.pitch_name, preceding.onset, target.pitch_name
ORDER BY preceding.onset ASC

From <https://gemini.google.com/app/57e91f4e4fcbe06e> Google Gemini (3.6 Flash)

Search This Blog

Learning from ChatGPT