The role of SQL or Spark pools (dedicated or serveless) in Azure Synapse

Question

1) What is the purpose of a SQL database or Spark pool (serverless or dedicated) in Synapse? Is it for acting as working storage for Synapse dataflows and pipelines? Or, is it for holding the final processed data to be consumed by analytical engine such as Azure Analysis Service or Power BI?

2) Can Azure Synapse Analytics be used without SQL or Spark pools?

3) Can Azure Synapse Analytics be used with Azure database instead of serverless or dedicated pools?

Answer 1

Hello @Suranga Manage ,

Welcome to the MS Q&A platform.

Serverless SQL pool:

Every Azure Synapse Analytics workspace comes with serverless SQL pool endpoints that you can use to query data in the Azure Data Lake (Parquet, Delta Lake, delimited text formats), Azure Cosmos DB, or Dataverse. A serverless SQL pool enables you to analyze your Big Data in seconds to minutes, depending on the workload.

Dedicated SQL pool :

Dedicated SQL pool (formerly SQL DW) refers to the enterprise data warehousing features that are available in Azure Synapse Analytics.
A dedicated SQL pool (formerly SQL DW) represents a collection of analytic resources that are provisioned when using Synapse SQL. The size of a dedicated SQL pool (formerly SQL DW) is determined by Data Warehousing Units (DWU).
Once your dedicated SQL pool is created, you can import big data with simple PolyBase T-SQL queries, and then use the power of the distributed query engine to run high-performance analytics. As you integrate and analyze the data, dedicated SQL pool (formerly SQL DW) will become the single version of truth your business can count on for faster and more robust insights.

Apache Spark pool:

Apache Spark is a parallel processing framework that supports in-memory processing to boost the performance of big data analytic applications. Apache Spark in Azure Synapse Analytics is one of Microsoft's implementations of Apache Spark in the cloud. Azure Synapse makes it easy to create and configure a serverless Apache Spark pool in Azure. Spark pools in Azure Synapse are compatible with Azure Storage and Azure Data Lake Generation 2 Storage. So you can use Spark pools to process your data stored in Azure.

for your the 1st question:
You can create external tables(don't actually hold the data, data is stored in data lake) using serverless SQL pools. There is no charge for resources reserved, you are only being charged for the data processed by queries you run, hence this model is a true pay-per-use model.

A dedicated SQL pool holds the data(like an Azure SQL server, data is stored in relational tables ). When the data is ready for complex analysis, a dedicated SQL pool uses PolyBase to query the big data stores. Cost is incurred for the resources reserved and pay per DWU provisioned.

2) Can Azure Synapse Analytics be used without SQL or Spark pools?
If you talking about SQL scripts or notebooks, you can't use them without SQL or Spark pools. (or) if your question is creating pipelines and dataflow, yes you can use Synapse analytics without using SQL or Spark pools.

3) Can Azure Synapse Analytics be used with Azure database instead of serverless or dedicated pools?

You can use Azure SQL DB as a source or destination in pipelines and dataflow but you can't use Azure SQL DB to run the SQL scripts in Synapse.

I hope this helps. Please let me know if you have any further questions.

---

• Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators