The accordion is a free reed instrument and is in the same family as other instruments such as the sheng and khaen. The sheng and khaen are both much older than the accordion and this type of reed did inspire the kind of free reeds in use in the accordion as we know it today.

The accordion's basic form is believed to have been invented in Berlin in 1822 by Christian Friedrich Ludwig Buschmann,although one instrument has been recently discovered that appears to have been built earlier.

The earliest history of the accordion in Russia is poorly documented. Nevertheless, according to Russian researchers, the earliest known simple accordions were made in Tula, Russia by Timofey Vorontsov from 1820, and Ivan Sizov from 1830. By the late 1840s, the instrument was already very widespread;together the factories of the two masters were producing 10,000 instruments a year. By 1866, over 50,000 instruments were being produced yearly by Tula and neighbouring villages, and by 1874 the yearly production rate was over 700,000. By the 1860s, Novgorod, Vyatka and Saratov Governorates also had significant accordion production. By the 1880s, the list included Oryol, Ryazan, Moscow, Tver, Vologda, Kostroma, Nizhny Novgorod, Simbirsk and others, and many of these places created their own varieties of the instrument.

The accordion is one of several European inventions of the early 19th century that used free reeds driven by a bellows. An instrument called accordion was first patented in 1829 by Cyrill Demian, of Armenian origin, in Vienna.

The piano accordion was first played in German-speaking regions, and then spread over Europe. Some early portable instrument with piano keys had been invented in 1821, but it started to actually be played much later, and built its reputation from there.

At that time in Vienna, mouth harmonicas with Kanzellen (chambers) had already been available for many years, along with bigger instruments driven by hand bellows. The diatonic key arrangement was also already in use on mouth-blown instruments. Demian's patent thus covered an accompanying instrument: an accordion played with the left hand, opposite to the way that contemporary chromatic hand harmonicas were played, small and light enough for travelers to take with them and used to accompany singing. The patent also described instruments with both bass and treble sections, although Demian preferred the bass-only instrument owing to its cost and weight advantages.

By 1831 at least the accordion had appeared in Britain. The instrument was noted in The Times of that year as one new to British audiences and not favourably reviewed, but nevertheless it soon became popular. It had also become popular with New Yorkers by the mid-1840s at the latest.

After Demian's invention, other accordions appeared, some featuring only the righthanded keyboard for playing melodies. It took English inventor Charles Wheatstone to squeeze both chords and keyboard together in one squeezebox. His 1844 patent for what he called a “concertina” also featured the ability to easily tune the reeds from the outside with a simple tool.

Jeune's flutina resembles Wheatstone's concertina in internal construction and tone color, but it appears to complement Demian's accordion functionally. The flutina is a one-sided bisonoric melody-only instrument whose keys are operated with the right hand while the bellows is operated with the left. When the two instruments are combined, the result is quite similar to diatonic button accordions still manufactured today.

Further innovations followed and continue to the present. Various buttonboard and keyboard systems have been developed, as well as voicings (the combination of multiple tones at different octaves), with mechanisms to switch between different voices during performance, and different methods of internal construction to improve tone, stability and durability.