This whole area was a little confusing at first. I had initially found a secret exit on stage 2, but I didn't follow that path and instead replayed the level to get the proper exit. The secret path seemed to disappear off the bottom of the screen.
After I'd played stage 3 and the tower, I found myself on the previously mentioned painted level (4) and another level (5). Each time I exited one of these levels, the path led back up in a loop. In order to progress, I actually had to find the secret exit in stage 5, which then took me down to meet the other end of the secret path from stage 2.