SQL Server does not really care about the format. If it is text data you can save the document in an nvarchar(MAX) or varchar(MAX) column. If the data is binary, such as an .docx file, you need to use varbinary(MAX). I believe RTF is a text format, but I am not sure. varbinary(MAX) will always work any way.
If you want to transform the document, you need to do this outside SQL Server, and for how to do it - this is the completely wrong place to ask.
SQL Server receives the generated bytes and that is what it returns. SQL Server never interpretes the bytes on its own account - with one exception: if you add a full-text index on a database, it is able to read PDFs, Word documents etc, provided that you somewhere store information about the format.