You'll need to check the list of the countries that have the visa exemption agreement with Japan. If your country is included in the list, then you will not be required for a visa when entering Japan under normal circumstances.

If your country is not included in the list, you'll need to obtain a short-term stay visa  from the embassy of Japan in your country.

Please read the Japanese visa guide provided by the ministry of foreign affairs carefully. And be informed that it is the embassy of Japan in your country that makes a decision on issuing a visa.  Whether the attendees need Letter of Invitation fully depends on the relations between countries. The conference organizers hereby reiterate that we are neither able to nor authorized to decide the relations between countries.

Invitation
Please be kindly noted that the organizing committee is not authorized to assist with the VISA process beyond providing the Invitation Letter. The invitation letter of conference organizing committee cannot guarantee the approval of visa application.

Invitation Letter
Please note the organizing committee of the conference is not authorized to assist with the VISA process beyond providing the Invitation Letter. Should your application be denied, the organizing committee of the conference cannot change the decision of the Ministry of Foreign Affairs, nor will the organizing committee engage in discussion or correspondence with the consulate or the visa center on behalf of the applicant.

The invitation letter will be issued upon request after the successful registration.