Will my store go down during the upgrade?

No. We use a blue-green deployment: the upgrade runs on a parallel staging environment, gets full QA, and then the DNS/load-balancer cuts over to the new version in under 30 seconds. Your live store keeps taking orders the entire time. We do schedule a 5-minute maintenance window for the final database sync, typically at 3 AM in your timezone.