Love this question!
Here's the story of the founding from the genius Jeff Atwood's blog.
They do not expound upon the name but I think it's fair to deduce that it is in fact, as you suppose, computer-related. This is especially likely given that the first Stack Exchange site, Stack Overflow, is concerned with programming languages and that Atwood's career is built upon programming.
But stack isn't so much a concept of programming as it is one of computer science. Some definitions are first needed. In computer science, a stack is a structured data set, which can be amended with a push or a pop, which are additions and subtractions from the dataset, respectively. The egalitarian nature of Stack sites means anyone can push data (in the form of questions, answers, suggestions, etc.) into the collective dataset, or stack. Only those with special privileges are able to pop, or remove from the stack.
Truthfully, I don't think it's the most appropriate metaphor. In computer science and especially in the rigorous methodology involved in maintaining complex systems stringent rules regarding, for example, a structure's form, data quality, chaining of events, and interrelationships are required. But it is an adequate metaphor, I think.