No clear successor emerged from political wrangling designed to decide which party would lead following the near three-way tie.
The Taoiseach, who has been in power since 2017, tendered his resignation on Thursday evening, RTE reported.
Mr Varadkar’s Fine Gael party came third behind Fianna Fail and Sinn Fein at the election two weeks ago.
Talks among the parties have failed to make much progress since the February 8 vote.
The election brought a breakthrough for Sinn Fein at the expense of the two long-dominant centrist parties, Fianna Fail and Fine Gael.
Fianna Fail and Sinn Fein each hold 37 seats in the 160-seat Dail, parliament’s lower house. Fine Gael has 35.
Thursday’s session opened with the election of a speaker, followed by parties nominating their candidates for Taoiseach, or prime minister.
Sinn Fein leader Mary Lou McDonald, Fianna Fail leader Micheal Martin and Mr Varadkar were all nominated to the top post.
However, none secured enough votes to be elected.
Following his resignation, Mr Varadkar will stay on as caretaker leader until a successor is chosen.
He is due to travel to Washington next month for the traditional St. Patrick’s Day visit to the White House by Ireland’s leader.
This page is being updated