When we had our 2nd, our 1st was not quite 2 1/2. I gave him a book from me (rather than from the baby) when I was in the hospital--Love you, Forever by Robert Munsch. He also got a baby doll (dressed in blue!) that he helped care for. (He's 7 now, and even though he doesn't play with that doll anymore, he still claims it as his and gets upset when either of his younger sisters wants to play with it!) I also had some small gifts for my older two when my 3rd was born, but I can't for the life of me remember what I gave them.
A couple of things that I think helped a lot were being willing to read to them or just let them snuggle up to you even when you're nursing/feeding, and also to let them help you with the baby care--fetch a diaper or wipes, sprinkle some powder on, etc.--to help them feel involved.
You might also check whether your hospital offers a "big brother" tour of the labor and delivery wing before you head there yourself.