I found out another way to do it.
https://graph.microsoft.com/v1.0/users/{from.aadObjectId}/drive/root:/{ATTACHMENTS.CONTENTURL.afterDocumentsPath}
With this, I have the reference to the object and a "@microsoft.graph.downloadUrl" link to use with axios and download its content.
const att = await graphClient.api("/users/a841a46f-80f6-41cd-8717-7e41d6817192/drive/root:/Microsoft%20Teams%20Chat%20Files/PDF.PDF").get();
const response = await axios.get(att["@microsoft.graph.downloadUrl"], { responseType: 'arraybuffer'});
const buffer64 = Buffer.from(response.data, 'binary').toString('base64');