You've Shown That It Is True, But You Haven't Explained Why It Is True. Therefore It Is False.

Good insight:

The reason "Who then created God?" is not a good objection is because the cosmological argument already addresses that issue. The whole point of these arguments is that there must be a cause that is not an effect of a previous cause itself. To ask why this first cause is this way is to ignore the argument that has just been made that this first cause is this way. Of course, showing that something is the case is not the same thing as showing why it is the case. (I would argue that one can answer the "why" question, but that's another issue.) But the atheist is claiming -- at least with this objection -- that unless the argument proves why something is the case, it doesn't prove that it's the case. This is obviously false.

So, for example, I could say that the ratio of a circle's circumference to its diameter equals pi (in Euclidean space). I could then prove this mathematically. The atheist objection would be "Why should this ratio equal pi?" The answer would be, "It does. Here's the proof again." The atheist would then object "Your mathematical proof doesn't explain why this ratio equals pi." And again, the answer would be, "It does equal pi. Here's the proof again." "But why should it be this way?" "It is this way. Here's the proof again." Etc. It reminds me of a Calvin and Hobbes cartoon where Calvin balks at his math homework. You put two numbers together and they magically become some third number. No one can say how or why it happens, you just have to accept it on faith. "As a math atheist, I should be excused from this."

So when an atheist asks why God should be excused from having to have a cause, the answer is simply to repeat the argument, which (allegedly) demonstrates that there must be a first cause that does not have a cause itself. Perhaps the argument fails to demonstrate this, but the objection that God would then require a cause doesn't even address it.