Why it's trickier than it looks
Dashi, a stock most commonly made from bonito flakes and kelp, underlies a huge share of Japanese cooking — miso soup, many noodle broths, and countless sauces — meaning a vegetable-only-looking dish can still carry fish stock at its base. This is the single biggest thing to be aware of, more than any single ingredient on a menu.
Useful phrases and signals
Asking specifically whether dashi is used, rather than just whether a dish is vegetarian, tends to get a more accurate answer, since some staff may not immediately connect a stock-based question with a broader vegetarian request. A growing number of restaurants, especially in central tourist areas, now carry English menus with allergy and dietary icons that make this easier.
Where it's genuinely easier
Buddhist temple cuisine (shojin ryori), where it's available, is reliably plant-based by tradition and worth seeking out for an easy, guaranteed vegetarian meal. Conveyor-belt sushi and ramen chains have both expanded vegetable and plant-based options in recent years, and konbini (convenience stores) increasingly stock clearly labelled vegetarian and vegan options for a reliable fallback.
A dedicated vegetarian or vegan restaurant-finder app is worth downloading before you land — the landscape changes often enough that an up-to-date list beats a static guide for exact addresses.