Gai was born as Hayk Bzhishkian in Tabriz, Iran, to a family of teachers. His mother was Persian and his father was an Armenian socialist (a member of the Armenian Social Democrat Hnchakian Party) who had taken refuge from the tsarist authorities in Persia during the 1880s. He returned to Russia in his teens and was an activist and journalist in Tiflis, where he studied at the Armenian Theological Seminary. He joined the Russian Social Democratic Labor Party in 1904 and spent five years in jail for revolutionary activities before he was drafted in 1914. Because of his background, Gai was assigned to the Russo-Ottoman front, where his repeated acts of bravery under fire earned him the rank of stabs-kapitan, the Cross of St. George (3rd and 4th class), and the Order of St. Anna, all awarded by General Nikolai Yudenich. Captured by the Ottomans, he escaped and returned to Russia badly wounded on the eve of the February Revolution. During World War I, Bzhishkian rose to the rank of captain. Gai, as he came to be known, became a Bolshevik before the October Revolution. He became a military commander in 1918 when he fought against the Czech Legion ("White Czechs") and the Orenburg Cossacks of Ataman Alexander Dutov.

During the Polish-Soviet War of 1920, he helped Mikhail Tukhachevsky drive the Poles back to Warsaw. Gai was appointed commander of the 3rd Cavalry Corps ("Kavkor"), attached to the 4th Army, and consisting of the 10th Division (N. D. Tomin) and the 15th (Kuban) Division (V. I. Matuzenko), with the 164th Rifle Brigade in support. In Tukhachevsky's plan, the role assigned to the Kavkor was "of the utmost importance". It was to operate on the extreme right wing of the Soviet advance and turn the flank of the Polish defense lines, thus allowing them to be rolled up by the attacking armies. The Kavkor advanced rapidly, taking Vilnius on July 14, Grodno (where the Red cavalry encountered tanks for the first time) on July 22, reaching the Vistula in the second week of August, and cutting the crucial Warsaw-Gdańsk railway.

However, the Polish counter-attack resulted in the encirclement of the 4th Army. Gai's Kavkor attempted to break out. After several engagements, it was finally pinned against the German (East Prussian) border by the pursuing Poles. The Kavkor crossed the border on August 26, and Gai was interned by the Germans in the Salzwedel camp near Berlin.

Gai was the People's Commissar of the Army and Navy of the Armenian SSR and later a military history lecturer and researcher in 1922. From 1924 to 1925, he was the chief of the military garrison in Minsk. In 1926, he continued his studies at the Military Academy of the General Staff. Upon graduation in 1927, Bzhishkian managed the Frunze department. He was a professor and the Head of the Department of War History and Military Art at the Zhukovsky Air Force Engineering Academy from 1933 to 1935.

He was twice awarded with the Order of the Red Banner; in 1919 for battles in the Volga Region of 1918 and in 1920 for the Polish campaign.

A monument dedicated to Gai stands in Nor Nork. Sculpture - Suren Nazaryan, Architect - Sargis Gurzadyan