I've been doing research on the 50 U.S. states and writing articles about them for years, and I can't remember ever seeing the word state capitalized when used by itself.
I consider the two sentences below correct:
"I was born in a state, not a province."
"I was born in the state of California."
This sentence is borderline...
"I was born in the State of California."
I think it's more appropriate when used in a legal or government context, e.g. a court representing the State of California. But would you write, "I was born in the Country of Japan"?
I think the rules for capitalizing the word president are a little ambiguous. I always capitalize it when it accompanies a name - President Carter, for example.
When the word president follows the word a, do not capitalize it...
When it follows "the," I think you could probably go either way...
"The president decided to back down."
"The President decided to back down."