No, you don't need to install all the PublishPress plugins.
You only have to install the ones you need. The same is true of most WordPress plugins.
The plugins will not slow down your site, but we find it very hard to imagine a site that needs them all.
For example, WooCommerce has over 300 add-ons available. Easy Digital Downloads has over 100 add-ons available. Most people will only install a few of those add-ons.
We recommend you take the same approach with PublishPress. Only install an plugin if you need the features it provides. If you're unsure if you need a particular plugin, you're welcome to contact us.