I have interviewed twice in scrubs, and got both jobs. BUT, these interviewers knew I was coming straight from work, and they worked with me to get the interview to fit both our schedules. When scheduling the interview, I mentioned that I wanted to be prompt, but needed to work my shift...so would it be ok to arrive in my scrubs, and they didnt care. They appreciated that I was honest, and still had that comittment to my other job. Of course if you have time to change, DO IT! But sometimes, your potential employer will appreciate you and your skills more than what you have on. In fact, my current job, I interviewed for in scrubs...and it happens to be a job where I am expected to look professional and deal with physicians and upper management often.
Again, if you have the time please change....but if it becomes an issue of thinking you may not make it on time, just let them know.