Earthquakes; tsunamis; typhoons; volcanic eruptions; landslides; floods. It's important to be prepared for these events when living in Japan. The resources below can help you be prepared. The most important thing is to stay connected to your contracting organisation, supervisor, local JETs, and local residents before, during and after natural disasters.
Japan Meteorological Agency - Keep up to date in English on all weather conditions and warnings throughout Japan.
Ready.gov - Ready.gov is a great resource for preparing for natural disasters, including this handy checklist to prepare your own disaster kit.
First Aid Kit (ファーストエイドキット)- Get a first aid kit or make one yourself. You have a few options here: Rakuten sells first aid kits, but you must have a credit card to do so (and sometimes it must be a Japanese card), Amazon Japan also sells them and you have the option of paying at a conbini, home supply stores (such as Tokyu Hands, Joy Home Center, Daiyu 8, etc.) also sell them. You may even be able to get first aid kits to keep in your car at car supply stores (such as Yellow Hat). You can also make one yourself by buying a convenient container at a 100 yen shop (like Daiso or Seria) and then filling it with your own supplies from a drug store such as Tsuruha or Sundrug (and sometimes also a 100 yen shop). The Red Cross has a list of what should go into a first aid kit here.
Many embassies offer emergency services to their citizens abroad. Their websites typically also include various tips and advice on what to do during an emergency event in Japan.