The dilemma of what to pack for a trip to Italy faces every traveler, but there are some specific things you might want to consider if you're packing for a train trip in Italy.
Everything is a Carry-On Bag
Whereas you might have checked an oversized suitcase on your flight to Rome, once you're taking the train all over Italy you'll be carrying your luggage on and off train cars yourself. There are some door-to-door luggage transport options, but they're limited to only the high-speed Frecce trains and only certain cities. The safest thing to do is assume that you will be responsible for carrying your bags on Italian trains at all times.
Traveling with a bag that is considered "carry-on size" by airlines, then, makes the task of climbing into and out of train cars much less physically taxing. Maneuvering smaller bags through crowded train stations and narrow train aisles is easier, too.
Maximize Luggage Space
When you limit yourself to traveling only with carry-on sized bags, you definitely have space restrictions that you don't have with full-sized luggage. The truth is that most travelers need far less than they actually bring—you can get away with wearing the same few outfits over and over (washing them periodically, of course), since no one will see you enough times to notice—so try to be ruthless when you're packing.
Leave bulky items at home, or plan to wear them on the plane so they don't take up space in your bags. Kindles, iPads or other e-readers are wonderful space savers for book lovers. Travel-sized toiletries may run out during your trip, but Italy has things like shampoo and toothpaste, too.
Carry a backpack or messenger bag (this can easily be your "personal item" on the plane) that's large enough to hold your valuables, any prescription medication, and maybe even a few items you picked up at the market for your on-board picnic. With this as your second bag, getting on and off trains is still easier than with a full-sized bag and you can keep all your valuables nearby even if you're stowing your luggage at one end of the train car.
Ship Souvenirs Home
Travelers who plan to shop while in Italy may start with excess room in an oversized suitcase, with the intention of filling it with souvenirs to bring home. There's nothing wrong with that, except the aforementioned issues with hauling heaving oversized bags on and off of trains.
Consider boxing up your goodies and shipping them from Italy to your home so you don't have to carry them or pack them. Some shops will take care of the shipping for you—especially if you're buying something large or expensive, such as wine or ceramics—so you won't even have to find a Mail Boxes Etc. in Florence.
Another option is to pack a duffel bag or another collapsible bag inside your suitcase—lay it flat and it shouldn't take up much room. You can expand into that bag as needed. It might make getting on and off of trains a bit more cumbersome with three bags (including that "personal item" bag), but some stations have trolleys you can pay to use.