The term krautrock was originally a humorous one coined by the UK music press (such as New Musical Express and Melody Maker), where "krautrock" found an early and enthusiastic underground following (It is unclear whether the term derives from the ethnic slur "Kraut", which had been used to refer to a German person in World War II, or whether it refers to marijuana, "Kraut" meaning "herb" in German). As is often the case with musical genre labels, few of the bands wished to see themselves pigeon-holed, and tended to eschew the term.

Julian Cope's book Krautrocksampler says "Krautrock is a subjective British phenomenon," based on the way the music was received in the UK rather than on the actual West German music scene out of which it grew. For instance, while one of the main groups originally tagged as krautrock, Faust, recorded a seminal 12 minute track they titled "Krautrock", they would later distance themselves from the term:

Krautrock is an eclectic and often very original mix of post-psychedelic jamming and moody progressive rock mixed with ideas from contemporary experimental classical music (especially composer Karlheinz Stockhausen, with whom, for example, Irmin Schmidt and Holger Czukay of Can had previously studied) and from the new experimental directions that emerged in jazz during the 1960s and 1970s (mainly the free jazz pieces by Ornette Coleman or Albert Ayler). Moving away from the patterns of song structure and melody of much rock music in America and Britain, some in the movement also drove the music to a more mechanical and electronic sound. The key component characterizing the groups gathered under the term is the synthesis of rock and roll rhythm and energy with a decided will to distance themselves from specifically American blues origins, but to draw on German or other sources instead. Jean-Hervé Peron of Faust says:

Typical bands dubbed "krautrock" in the 1970s included Tangerine Dream, Faust, Can, Amon Düül II, Ash Ra Tempel and others associated with the celebrated Cologne-based producers and engineers Dieter Dierks and Conny Plank, such as Neu!, Kraftwerk and Cluster. Bands such as these were reacting against the need to develop a radically new musical aesthetic and cultural identity for the post-WWII. Many of these groups began their musical careers with little or no awareness of (or interest in) rock and roll: exposure to the increasingly radical and innovative music of the Velvet Underground, the Silver Apples, Frank Zappa, Jimi Hendrix and the Beatles, for example, led members of groups like Can and Kraftwerk to embrace popular music for the first time.

The signature sound of krautrock mixed rock music and "rock band" instrumentation (guitar, bass, drums) with electronic instrumentation and textures, often with what would now be described as an ambient music sensibility. A common rhythm featured in the music was a steady 4/4 beat, often called "motorik" in the anglophone music press.

The term is arguably xenophobic in tone due to the word Kraut having a history as an insult for German people.. It is also a problematic category due to the considerable differences between the artists so labelled. Acts such as Can, Neu! and Kraftwerk have little in common audibly.

By the end of the 1960s, the American and British counterculture and hippie movement had moved rock towards psychedelia, heavy metal, progressive rock and other styles, incorporating, for the first time in popular music, socially and politically incisive lyrics. The 1968 German student movement, French protests and Italian student movement had created a class of young, intellectual continental listeners, while nuclear weapons, pollution, and war inspired protests and activism. Avant-garde music had taken a turn towards the electronic in the mid-1950s. The avant-garde minimalist music current which emerged in the beginning of the 60s with the works of Americans La Monte Young, Terry Riley and Steve Reich using drones and loops (often with synthesizers and tapes) in a kind of psychedelic and space-oriented music.

These factors all laid the scene for the explosion in what came to be termed krautrock, which arose at the first major German rock festival in 1968 in Essen. Like their American, British and international counterparts, German rock musicians played a kind of psychedelia. It was however, strikingly innovative as a fusion of psychedelia and the electronic avant-garde. That same year, 1968, saw the foundation of the Zodiak Free Arts Lab in Berlin by Hans-Joachim Roedelius, and Conrad Schnitzler, which further popularized the psychedelic-rock sound in the German mainstream. Originally krautrock was a form of Free art, which meant that krautrock bands gave their records away for free at Free Art Fairs.

The next few years saw a wave of pioneering groups. In 1968, Can formed, adding jazz to the mix (and in that way the krautrock scene can be seen to parallel the emerging Canterbury scene in England at the same time), while the following year saw Kluster (later Cluster) begin recording electronic instrumental music with an emphasis on static drones. In 1970, Popol Vuh became the first krautrock group to use an electronic synthesizer, to create what would be known as "kosmische musik". The bands Tangerine Dream, Ash Ra Tempel, and Cosmic Jokers (all linked by collaboration with Klaus Schulze), would follow suit in the years to come. Faust also made use of synthesizers and tape manipulation in a way foreshadowing the noise rock of the future.

In 1972, two albums incorporated European rock and electronic psychedelia with Asian sounds: Popol Vuh's In den Gärten Pharaos and Deuter's Aum. Meanwhile, kosmische musik saw the release of two double albums, Klaus Schulze's Cyborg and Tangerine Dream's Zeit (produced by Dieter Dierks), while a band called Neu! began to play highly rhythmic music. By the middle of the decade, one of the best-known German bands, Kraftwerk, had released albums like Autobahn and Radioaktivität ("Radio-Activity" in English), which laid the foundation for electro, techno and other styles later in the century.