My gardens have always been slow and steady. They're never first to produce and they never have huge plants. But they're often some of the last to remain in production and resist disease. It could be because I genuinely don't pay plants much attention once they start growing. I don't water frequently, and I don't fuss with them much. I like to look at them and observe what they're doing and what bugs are on them, but I don't coddle them. Which is not necessarily the best way to do things. Coddled plants produce a lot. But they also demand a lot of your attention!
So if your plants are small, that's ok. They will probably just produce later than plants that are currently huge. If they're sick and tiny, they may die. But if they're small and perfectly healthy, they will probably be fine.
Reasons your plants might look sick: if they're yellowing, they need fertilizer (see next paragraph). If they're crispy brown at the ends of the leaves, they're experiencing drought. Water more deeply. If they're wilting in the heat of the day, they should perk up at dusk. If not, water them deeply, and if they keep looking wilty, there's something else wrong with them that no amount of water can fix. If they've got a million holes, take a close look at what bugs are sitting on the leaves and google them. Vegetable diseases are many, and it's beyond the scope of my expertise to diagnose them.
I never used fertilizer before I started working at Natureworks, and now I would not garden without some form of it. It just makes your plants come out better and leaves you feeling more satisfied. I like Neptune's Harvest Tomato and Veg formula. It has a red label. Of course I don't fertilize nearly as much as I should, but it still does the plants good, and it's the only way I can get broccoli to head.
Below is a small tomato plant. Maybe it will fruit properly. Maybe it won't!