The "cave" is actually an underground drainage system which runs from Old High to the Country Club.
Chief Holzer said the children were swept underground by the water.
A short time later, one boy crawled out of a curbside drain.
His sister re-surfaced a few hundred yards downstream in the Martin Plaza area.
Rescue crews were searching that area looking for a 13-year-old boy.
That's when Holzer said police located him near the area where the incident started.
"It's just one of those things where we're thanking our lucky stars it's not a recovery, and it ended well," said Chief Holzer.
Authorities said the 13-year-old was taken to United Regional for treatment of possible hypothermia and shock, but he and the other children appear to be okay.