After playing in a variety of swing orchestras, including Lloyd Hunter's Serenaders, he founded his own band in 1945 and had one of the most enduring hits of the big band era, "Harlem Nocturne". This band played with Wynonie Harris and Charles Brown. In 1947, he and Bardu Ali opened the Barrelhouse Club in the Watts district of Los Angeles. He reduced the size of his band and hired singers Mel Walker, Little Esther Phillips and the Robins (who later became the Coasters). He discovered the teenaged Phillips when she won one of the Barrelhouse Club's talent shows. With this band, which toured extensively throughout the United States as the California Rhythm and Blues Caravan, he had a long string of rhythm and blues hits through 1950.
In the late 1940s, he discovered Big Jay McNeely, who then performed on his "Barrelhouse Stomp". In the 1950s he discovered Etta James, for whom he produced her first hit, "Roll With Me, Henry", (also known as "The Wallflower"). Otis produced the original recording of "Hound Dog" written by Jerry Leiber and Mike Stoller with vocal by Big Mama Thornton, and was given a writing credit on all six of the 1953 releases of the song. As an artist and repertory man for King Records he also discovered Jackie Wilson, Hank Ballard, and Little Willie John, among others. He also became an influential disk jockey in Los Angeles. However, he continued to perform, and in April 1958, he recorded his best-known recording "Willie and the Hand Jive", which went on to be a huge hit in the summer of 1958, peaking at #9 on the U.S. Pop chart, becoming Otis' only Top 10 single. His most famous composition is "Every Beat of My Heart", first recorded by The Royals in the 1952 but which became a huge hit for Gladys Knight.
Otis continued performing through the 1990s and headlined the San Francisco Blues Festival in 1990 and 2000, although because of his many other interests he went through long periods where he did not perform.
Frank Zappa has cited Otis as the inspiration for his distinctive trademark facial hair, stating in an interview conducted by Simpsons creator Matt Groening and Guitar Player magazine editor Don Menn, "It looked good on Johnny Otis, so I grew it."