Frequently asked questions about testing your Business Central App
This unit contains answers to frequently asked questions about testing your app when you submit an app to AppSource for Business Central.
Since my app goes through validation, do I really need to thoroughly test my app?
Yes. It is your app and you are the expert, so you should have a vested interest in high quality. The main goal of validation is to ensure app code is meeting requirements and policy. The testing done during validation is only to ensure a good user experience on the most common app scenarios. We don't test every scenario of an app. Using customers to find bugs isn't the proper approach.
Is it ok to submit my app for validation before we complete our own testing?
No. Don't submit your app for validation until it has been 100% tested. This will typically lead to many delays in the validation process otherwise. Also, it wastes time and resources on both ends.
Is it ok to test my AppSource app in an on-premises environment?
No. You must test your app using Business Central online. Although Business Central online and on-premises are similar, they are not exactly the same. If you only test on-premises, invariably issues will be found in our validation testing.
Does it really matter what version of Business Central I test on before submitting my app for validation?
Yes. It's critical that you always test on the latest version at the time when you are ready to submit for validation. Testing on the wrong version usually leads to validation failure. For example, let's say the latest version of Business Central is 15.4 at the time when you submit for validation. You tested on 15.0. The product has changed between those versions, and deprecated features or other changes could result in your app behavior changing.
We recommend that you consider using Docker deployments. The Current image tag removes the guessing. You don't have to figure out what product version is active at time of submission. That image tag does that for you.
How thorough should our testing be?
You should always test with 100% coverage, or at least as close to 100% as you can get. The testing should be a combination of automated and manual tests. The apps with good testing infrastructure behind them are the most successful.
Do I have to test on every country/region that I intend to support with my app?
Yes. If you support multiple countries/regions, test your app for each country/region. Each country's/region's code base is slightly different from both the base application and other countries/regions. It's critical to make sure that the app publishes, syncs, and installs on every country/region you support. Because an app installs fine for one country/region, doesn't mean it will publish/install fine on another. We've seen many times where it passes on one country's/region's installation, but fails on another during the validation process.
Do you have recommendations on maintenance testing of our apps?
Yes. You should be testing your apps against our various build branches. Through Docker, you have access to our Current, Next Minor, and Next Major builds. Test often, especially against the Next Minor branch. This allows you to catch any bugs that may arise from core changes in the product.
I only made minor code changes in my updated app. Can I test just these changes?
No. You should always test 100% coverage no matter what. Testing only what you changed is not the correct approach. Even minor changes can lead to breaking changes, often where you least expect it.
Do I need to do upgrade testing?
Yes, this is a must. If an app fails to upgrade, customers are unable to get your updated app. This failure is one of the common failures we see in validations. You should want your app upgrade to work optimally.
Any recommendations on upgrade testing?
Test the upgrade with extensive app data included. Many of the upgrade failures for customers are data-related. The upgrade fails because specific data scenarios were not considered for testing. Or worse, the upgrade succeeds, and data is lost.
Do I only need to do upgrade testing from previous version to current?
No. It is important you test from various previous versions of your app. This is because we don't automatically upgrade apps for minor releases. You could have a tenant back on version 1.0.0.0 of your app, and must jump all the way to version 1.0.0.5. We don't guarantee direct upgrades of apps from their most previous version.