Deployment of Etherpad-Lite an open source node.js project using symbolic links

Question

I'm trying to set up Etherpad-Lite in an App Service. This is an open source Node.JS project, so it can go into a Linux or Windows server -- I'm trying to stick with Windows since that's what my team and I know best.

I'm running into issues with symbolic links. I understand that Azure doesn't give permission to create symlinks via the console, but I've read that it will accept symlinks as part of a deployment.

I've tried to deploy via 4 methods:

FTP: Transferring the files via FTP breaks the symlinks, so the site has runtime issues.

VS Code: Deploying via VS Code creates a ZIP file (which I suspect will break the symlinks) that fails extraction on the remote server with "There is not enough space on the disk."

Github: I cloned the project, made my configuration changes, and then attempted to deploy via Github, but I can't get the deployment to start. Clicking on Redeploy/Sync immediately reports a successful deployment, but no changes are made to the file system.

Azure Repo: Same behavior as a Github deployment.

The code runs fine locally on Windows 10 (with developer mode enabled). I also set up an Ubuntu Hyper-V VM (just in case using Linux would fix it). It runs fine in Linux and deploying via FTP or VS Code on Linux also fail in the same way.

So, at this point I've tried so many things that I'm happy to restart anywhere you think would be a good place. I also posted these questions on the Azure subreddit: https://www.reddit.com/r/AZURE/comments/lm5l6x/has_anyone_set_up_etherpad_in_azure/

I guess I have three questions to start with:

1) Which deployment methods support deploying symbolic links? Or is there some way to recreate them? I've only successfully deployed with FTP.
2) Should I deploy node_modules, or is there a reliable way for the server to retrieve the modules after my first deployment?
3) Is there a better technology to use than an App Service?

Edit: The Etherpad Lite project page: https://github.com/ether/etherpad-lite

Answer 1

I'm making progress with a Github deployment. It's now starting a workflow in Github to do the deployment. I'm running into an issue with the package.json file. I think it's looking for this file in the site root, but it's located in a subfolder, src.

Is it normal to have the package.json in a non-root folder? How can I tell npm where to find this file?

Here's the relevant output of the workflow.

Run npm install
  npm install
  npm run build --if-present
  npm run test --if-present
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
npm WARN saveError ENOENT: no such file or directory, open 'D:\a\accountname\webappname\package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open 'D:\a\accountname\webappname\package.json'

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path D:\a\accountname\webappname\package.json
npm ERR! errno -4058
npm ERR! enoent ENOENT: no such file or directory, open 'D:\a\accountname\webappname\package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

Answer 2

Hey @Ryan Hill ! Thanks for your message.

I'll hunt for the article, but it was in a Microsoft forum or a StackOverflow post, so it definitely might not have been reliable.

I need to use symlinks because the software package that I'm trying to deploy (etherpad-lite) is coded to use symlinks. To be honest, I don't know why the developers used them, but it's a Node.JS project and it seems to be a common enough occurrence with Linux development? I do know that etherpad-lite uses npm at runtime to install plugins, so perhaps that's the reason? In any case, I personally do not care if the symlinks are "baked" and all of the files duplicated.

I've gotten a bit further since my last post. I didn't realize that the above error messages where from the workflow on Github's side. I fixed those issues and the deployment succeeds. The symlinks were not copied, as expected.

The application code starts. In a web browser, it returns HTTP 500.1001 error: iisnode was unable to establish named pipe connection to the node.exe process. And that's all that I can learn from the Log Streams via the Azure web portal (diagnostic logs are turned on). I downloaded the data dump via https://<app-name>.scm.azurewebsites.net/api/dump and found the application logs:

 [36m[2021-02-17 20:38:22.572] [DEBUG] console -  [39mRunning on Node v12.13.0 (minimum required Node version: 10.17.0)  
 [32m[2021-02-17 20:38:22.759] [INFO] console -  [39mAll relative paths will be interpreted relative to the identified Etherpad base dir: C:\home\site\wwwroot  
 [32m[2021-02-17 20:38:22.759] [INFO] console -  [39mRandom string used for versioning assets: b1e64664  
 [36m[2021-02-17 20:38:22.775] [DEBUG] AbsolutePaths -  [39mRelative path "settings.json" can be rewritten to "C:\home\site\wwwroot\settings.json"  
 [36m[2021-02-17 20:38:22.791] [DEBUG] AbsolutePaths -  [39mRelative path "credentials.json" can be rewritten to "C:\home\site\wwwroot\credentials.json"  
 [32m[2021-02-17 20:38:22.827] [INFO] console -  [39msettings loaded from: C:\home\site\wwwroot\settings.json  
 [32m[2021-02-17 20:38:22.838] [INFO] console -  [39mNo credentials file found in C:\home\site\wwwroot\credentials.json. Ignoring.  
 [32m[2021-02-17 20:38:22.838] [INFO] console -  [39mUsing skin "colibris" in dir: C:\home\site\wwwroot\src\static\skins\colibris  
 [32m[2021-02-17 20:38:22.853] [INFO] console -  [39mSession key loaded from: C:\home\site\wwwroot\SESSIONKEY.txt  
 [31m[2021-02-17 20:38:37.541] [ERROR] console -  [39m(node:14704) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.  
 [32m[2021-02-17 20:38:39.855] [INFO] runNpm -  [39mnpm --version: 6.14.11  

At this point those logs end without any errors reported (other than the Buffer() ERROR, but this happens on the dev machines too and shouldn't be fatal).

The developer's thoughts:

Are you sure that it stops? The line in the log that says "runNpm - npm --version: 6.14.11" is printed just as Etherpad runs npm to discover installed plugins. The npm CLI is slow, so maybe it's just taking a while? It should take less than 60s to return.  
If it is indeed stopping but the logs don't show anything, I wonder if it's a Windows-specific log buffering issue. Maybe the process is dying and Windows isn't flushing buffered writes so you can't see the printed error message.  

You can find this conversation and the Github debugging here: https://github.com/ether/etherpad-lite/issues/4800

What do you think? Could there be a logging issue? Is there a better place to get the logs from? I'm not surprised that it's crashing because I expect that there are application configuration issues, but the silent failure makes me think that there's a larger issue to correct.