Well, it all depends; I say, Plaza Mayor is a good place to stay but, in my opinion, rooms with kitchen are a bit nasty over there. If I had to get a baby, I´d prefer a good place to stay (just the contrary than Gran Vía; the downtown is really noisy at night just because the buildings are really old and the windows are made of soft glass) and, if you don´t want to walk a lot, up & down, from site to site, you can get the Metro - perfect to get quickly in everywhere.
The trip between Atocha to Gran Vía-Sol-Plaza Mayor is about 30 minutes by subway, so It´s comfort for your baby and you´ll be close to the station the day you travel to Alicante.
Enjoy a lot!!
(More questions or doubts are wellcome)