Sending data frame as an attachment in email using synapse notebook (PySpark)?

Question

Sending data frame as an attachment in email using synapse notebook (PySpark)?

Satheesh K 45

Hello,

Is it possible to send pyspark dataframe as an attachment in email using synapse notebook (PySpark)?

Thanks

Ganesh Gurram 6,040 Reputation points Microsoft External Staff

2024-10-08T04:11:40.3+00:00

@Satheesh K - We haven’t heard from you on the last response and just checking in to see if the below answer helped. If this answers your query, do click "Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

1 answer

Your answer

Ganesh Gurram 6,040 Reputation points Microsoft External Staff

2024-10-08T04:11:40.3+00:00

@Satheesh K - We haven’t heard from you on the last response and just checking in to see if the below answer helped. If this answers your query, do click "Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Answer 1

@Satheesh K - Thanks for the question and using MS Q&A plaform.

You can send a data frame as an attachment in an email using a Synapse notebook with PySpark.

Here's an example code snippet that demonstrates how to do this:

from email.mime.text import MIMEText
from email.mime.application import MIMEApplication
from email.mime.multipart import MIMEMultipart
from smtplib import SMTP
import smtplib
import sys
import pandas as pd

df_test = pd.read_csv('abfss://******@samplesynapseadlsgen2.dfs.core.windows.net/Data/moviesDB.csv')

email_user = '******@gmail.com'
email_password = 'PASSWORD'

recipients = ['******@gmail.com'] 
emaillist = [elem.strip().split(',') for elem in recipients]
msg = MIMEMultipart()
msg['Subject'] = 'SUBJECT'
msg['From'] = '******@gmail.com'


html = """\
<html>
  <head></head>
  <body>
    {0}
  </body>
</html>
""".format(df_test.to_html())

part1 = MIMEText(html, 'html')
msg.attach(part1)

server = smtplib.SMTP('smtp.gmail.com', 587)
server.starttls()
server.login(email_user,email_password)
server.sendmail(msg['From'], emaillist , msg.as_string())

Here is the screenshot of the code which I had ran on my synapse notebook:
User's image

Here is the email sent using synapse notebook (PySpark):
User's image

In case, if you are experiencing this issue: SMTPAuthenticationError: (535, b'5.7.8 Username and Password not accepted. For more information, go to\n5.7.8 https://support.google.com/mail/?p=BadCredentials 98e67ed59e1d1-2e1e8625ccbsm714495a91.48 - gsmtp')
User's image

Reason: It will throw this error message if 2-Step Verification turned on.

App passwords can only be used with accounts that have 2-Step Verification turned on.

If you do not have a gmail apps password, create a new app with using generate password. Check your apps and passwords https://myaccount.google.com/apppasswords

For more details, refer to SO thread: smtplib.SMTPAuthenticationError: (534, b'5.7.9 Application-specific password required addressing similar issue.

For more details, refer to SO thread: Is there a way to send email with a dataframe attachment? addresses similar issue.

Hope this helps. Do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Satheesh K 45 Reputation points

2024-10-04T14:44:37.8633333+00:00

We are getting connection timeout on the below command. Is there any specific network setting that should be enabled?

server = smtplib.SMTP('companysmtpserver', 587)
Ganesh Gurram 6,040 Reputation points Microsoft External Staff

2024-10-07T10:35:16.7266667+00:00

@Satheesh K - Did you tried the above script to send a data frame as an attachment in an email using a Synapse notebook with PySpark?

In case, if you are using a different script, please do share the script which you are trying so that we can help you.

There could be a number of reasons why you are getting a connection timeout error. Here are a few things you can check:

Firewall: Make sure that the firewall on your network is not blocking outgoing traffic on port 587. Port 587 is the default port used for SMTP submission, so it needs to be open for your email client to connect to the SMTP server.

DNS: Ensure that your DNS settings are correct and that you can resolve the hostname of the SMTP server. You can use the nslookup command to check if the hostname is resolving to the correct IP address.

SMTP server: Check if the SMTP server is up and running. You can use the telnet command to test if you can connect to the SMTP server on port 587. For example, telnet companysmtpserver 587.

If none of these steps help, you can try using the tcpping tool to test connectivity to the SMTP server. The tcpping utility tells you if you can reach a specific host and port. It can show success only if there's an application listening at the host and port combination, and there's network access from your app to the specified host and port. The syntax for tcpping is:

tcpping.exe hostname [optional: port]

If you are still having issues, please provide more information about your network setup and the error message you are seeing.
Ganesh Gurram 6,040 Reputation points Microsoft External Staff

2024-10-09T05:25:40.9433333+00:00

@Satheesh K - We haven’t heard from you on the last response and just checking in to see if the above answer helped. If this answers your query, do click "Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Share via

Sending data frame as an attachment in email using synapse notebook (PySpark)?

1 answer

Your answer