Here is an interesting recent PW thread on playing by ear. The response that really caught my eye was from Gyro. Below I quote in full what he wrote. Something simple to start with and to build on. No music theory needed! 🙂
Practical by-ear playing is not that difficult. Starting on the C one octave below middle C, and going up the white keys, play the 7 four-note chords using every other white key:
CEGB, DFAC, EGBD, FACE, GBDF, ACEG, BDFA.
This is the foundation upon which all chords for by-ear playing are built. (In some songs, particularly older songs and rock and country oldies, sometimes only 3-note chords are used: CEG, DFA, EGB, etc.)
Note that each of the 4-note chords above can easily slip into a second chord without changing the first two notes: for example, CEGB can slip easily into CEFA, or CEGA, etc. This is very easy to play on the piano, because the fingers that played the first two notes of the first chord remain in place and only the other finger or fingers move to slip into the notes of the second chord. When you play such a two-chord sequence, like CEGB/CEFA, you’ll note that the sound is very familiar: this is the “ta-dah” pattern that you hear so often in all the music we are familiar with.
Similarly, DFAC slips into DFGB, or DFAB, etc. And so forth. When playing by ear, when you determine that, say, the chord DFAC fits with the melody, then the next chord is often one that DFAC slips easily into, that is, DFGB, or DFAB.
Note that each of the 7 four-note chords above can be easily changed into a chord that differs by only a few notes, for example, CEGB can be changed into CEbGBb or C#EG#B, etc. Thus, in and around those 7 chords are all the chords you would need to play any song by ear.
Note the ear playing is not an exact science, and there are no absolutely correct chords for any song: the chords or melody in your by-ear version might differ considerably from the commercial sheet music version of the song–that’s okay by ear-playing standards.
To give some practical examples, say you want to play Way Down Upon the Swanee River without looking at a sheet. You can pick out a melody (try to stick with white keys mostly because our 7 base chords are from C maj., all white keys): EDCEDCC (one octave up) ACGECD (the first E is an octave above middle C). To harmonize this, note that is is an older song so the chords will be mostly 3-note chords instead of jazz-style 4-note chords. This is done by experimenting by ear, as there is no exact formula for doing it: CEG (the C is an octave below middle C), played together with the first E of the melody; EGC (this is CEG inverted one time–the C is put one octave higher–you can often simply invert a chord instead of looking for a completely new chord, or for filling in, as is the case here), played with the second D of the melody; GCE (this is CEG inverted twice–the C and the E are put an octave higher), played with the second E; FAC played with the second C; ACF (FAC inverted) played with the first A; CEG played with the first G; EGC (inversion) played with the third E; GBDF (a four-note chord) played with the third D; FGBD (GBDF inverted downwards) played after the previous chord, with no note in the melody. And so forth by ear.
Suppose you want to play Misty without looking at a sheet. Picking out the melody by ear:
Since this is a modern jazz-type song the chords will be 4-note chords. Experimenting by ear, it seems that this song can be played with a lot of inversions, for the first stanza: CEGB played with the first B; EGBC (inversion of CEGB) played between the first B and the second G, no melody note; GBbDF played with the first Bb; FGBbD (downward inversion) played with the fourth A; FACE played with the fifth A; EFAC (downward inversion) played after the fifth A, no melody note.
For the second stanza: FACE played with the first A; DFAC played with the second G; FACD (inversion) played with the fifth G; EGBD played with the second E; GBDE (inversion) played with the second C; DFAC played with the fourth F; FACD (inversion) played with the first B;
EGBD played with the fourth E; GBDE played after the fourth E, no melody note. Etc.