English has no systematic way of naming people of a given nationality. Some nationalities have only an adjective ("he is a Chinese man"), some have an adjective and a special noun ("she is a Scottish woman/she is a Scot", "He is a Swedish man/He is a Swede"), and some have an adjective and a noun that are the same word ("she is a German/she is German")
In your question, you've discovered the first category (Japanese) and the third (German), and you've also touched on the only real rule in deciding between them...
"-an": the noun is (usually) the same
Where the adjective ends with "-an", you can be reasonably confident in using it as a standalone noun too. For example, the following are all usable as the word in the two sentences "She was a(n) _____ woman" and "She was a(n) _____":
Be careful that you don't accidentally a word that has a more common meaning already: someone from the Dalmatia region of Croatia is indeed "a Dalmatian", but if you use that word, people will think you're calling them a dog.
"-ese" can't be used as a noun
For adjectives ending in "-ese", you can't use the word as a noun. There's either a special noun to describe the person (Faroese/Faroer), or none at all (Chinese, Japanese). There aren't many of these adjectives in English.
everything else, you have to learn, sorry.
In cases where the adjective doesn't end in any of these, it's hit or miss.
Not "-an", but still the same word
In many cases, the "official" noun is the same as the adjective, although this is an old rule, and today most of these will sound a little odd, archaic, or even insulting even though they're still technically correct (e.g., "an Argentine","a Basque", "a Swiss") - the only one I can think of offhand that doesn't sound strange is "Greek".
"A Basque" is also an example of the case where you need to be careful of not using a word that has another more common meaning (here, the kind of corset worn by women).
use the stem
There's also a whole collection of special nouns, like "Swedish/Swede", "Finnish/Finn" where the noun is just the stem of the adjective. But be careful: "a Flemish man " is most definitely not a "Flem"