Posted by nathancraig on Mon, Nov 30, 2020 - 4:32pm

I'm trying to update both core and some modules via the web interface.

https://example.org/admin/appearance/update

My webhost requires SFTP connection. However, through the Backdrop web interface linked above I only see FTP as a connection option. When I use working credentials in the web update interface, I get an error saying that I can't connect to the FTP server.

Is this likely an issue that the server only accepts SFTP connections and not FTP connections? If that is the case, are there plans afoot to perform web interface updates via SFTP? It would be really handy to be able to just click and button and fill in credentials to perform an update. Thanks in advance.

Most helpful answers

If I got it correctly,

If I got it correctly, nathancraig tried to use the UI ("through the Backdrop web interface linked above"). But depending on the server configuration (at least in Drupal) it can happen that you have to put FTP credentials while updating a module via the user interface.

I guess the solution is to configure the server in a way that you don't need (S)FTP credentials. Here's a short article about it (regarding Drupal):

https://stevenschwenke.de/updatingModulesAndThemesRequiresFTPAccessToYou...

I don't have an answer to

I don't have an answer to your specific question, but have you tried using the Project Installer module to do the updates directly in the UI without needing SFTP credentials? See a demo here (sorry the audio quality is not great):

Comments

Olafski's picture

Tue, Dec 01, 2020 - 2:53am by Olafski

If I got it correctly, nathancraig tried to use the UI ("through the Backdrop web interface linked above"). But depending on the server configuration (at least in Drupal) it can happen that you have to put FTP credentials while updating a module via the user interface.

I guess the solution is to configure the server in a way that you don't need (S)FTP credentials. Here's a short article about it (regarding Drupal):

https://stevenschwenke.de/updatingModulesAndThemesRequiresFTPAccessToYou...

Tue, Dec 01, 2020 - 5:32pm by nathancraig

Thank you for the leads. The first video illustrates what I'm trying to do.

1. Under updates, settings, Manual self-update is checked and saved.

2. When I go to updates modules and select an item or items to update, and proceed to download these updates, I get a continue button asking me to put the site into maintenance mode. Following this, I come to a screen not shown in the youtube link you shared, but one that is shown in the article you linked.

3. From the article, I gather than I must "change the owner of sites/default to the PHP-user (www-data)".

I read the drupal docs linked in your article and searched the page for php. I saw several mentions of the term but nothing on setting the owner of www-data.

I'm confused because the user who owns the directory where backdrop is stored is different from the PHP-user that accesses the database. In other words, the web host doesn't have a means to sign into the server with the PHP-user--they are different logins. I may be misunderstanding things. I thank you for your suggestions, but do you have any further leads or can you explain further what you mean by "change the owner of sites/default to the PHP-user (www-data)". For example, to the best of my knowledge, I don't have a www-data directory.

Olafski's picture

Wed, Dec 02, 2020 - 12:47am by Olafski

Unfortunately, I can't help much with server configuration questions. As the issue can happen in Drupal as well, and Drupal has more installations in the wild than Backdrop, maybe search the web for something like "drupal update module ftp credentials" and adapt the results to Backdrop.

In one of the first results (https://www.drupal.org/forum/support/post-installation/2011-01-22/module...) the www-data question is explained in different words:

To avoid this, make sure the folder /sites/default is OWNED by the user that executes the drupal scripts. On most Ubuntu installations, this is the apache user: www-data.

So, the user doesn't have to be www-data. The name seems to depend on the operation system. Also, usually (or: out of the box) there is no /sites/default directory in Backdrop.

I guess my answer doesn't help directly, so I hope someone with more server configuration skills can chime in.

Sat, Dec 05, 2020 - 10:51am by nathancraig

This is helpful. I've contacted the admins with the web host. I will report back what I learn. Much appreciated.

themetman's picture

Wed, Feb 03, 2021 - 2:29am by themetman

I have been having the same problem updating a couple of modules from the GUI. It used to work fine on my Gentoo System, not asking about FTP, but I have switched to Linux Mint, and now it will not work.

The FTP seems new for some reason (I have just gone to Backdrop 1.18.1). I have enabled the ftp module like so

sudo a2enmod proxy_ftp

But then I get an error:

views_slideshow_cycle2
Error installing / updating
File Transfer failed, reason: /var/www/html/wlg.local/web/modules/views_slideshow_cycle2 is outside of the /var/www/html/wlg.local/web

I have my site outside the web root at /var/www/html/wlg.local/web/

So, how can I overcome this, I think it is looking to download somewhere before overwriting the current module.

I have looked here and here  and here but none of this helps.

By the way, installing new modules works prefectly.

Any ideas?

 

themetman's picture

Wed, Feb 03, 2021 - 7:49am by themetman

Thanks @indigoxela I suspect it might be. I have a symlink like this:

/var/www/html -> /home/FG-Docs/public_html/

Then I have the docroot as:

/var/www/html/wlg.local

with the backdrop core in

/var/www/html/wlg.local/web/

I will give it a try with the actual path in a little while and see if that is OK

themetman's picture

Thu, Feb 04, 2021 - 4:14am by themetman

That made no difference, @indigoxela, so I will just have to do it manually for now.

themetman's picture

Mon, Feb 08, 2021 - 6:53am by themetman

@indigoxela I have sorted it.

As soon as I changed the permissions on the folders to the same as the Apache user, everything worked fine.

Regards

indigoxela's picture

Mon, Feb 08, 2021 - 7:09am by indigoxela

Glad that you sorted things out! Enjoy using Backdrop. :-)

Fri, Mar 26, 2021 - 12:52pm by cptX

Hi, I have exactly the same problem in linux mint using nginx. Can you please describe how you solved it? What file permissions and ownership did you give to the file structure?

Fri, Mar 26, 2021 - 12:42pm by cptX

Hi I have exactly the same problem in linux mint using nginx. @themetman can you please describe how you solved it? What file permissions and ownership did you give to the structure?

 

Wed, May 26, 2021 - 3:54pm by brad.bulger

I am getting this FTP server stuff trying to update my local copy to 1.19.1. The docroot is not a symlink, all the file ownership the same as ever, this has worked with prior releases.

 

UPDATE: sorry, the Apache DocumentRoot is set to a symlink. So I guess this is the same bug as ever. I swear I was able to make this work before, though. Maybe not.

 

Wed, May 26, 2021 - 4:08pm by brad.bulger

Now I see on /admin/modules/update this line

"Updating modules, themes, and layouts requires FTP access to your server. See the handbook for other update methods."

Is this new? It won't update directly on the installation, but only download and then upload via FTP? It is literally running on the same computer I am running the browser in, that seems like a peculiar way to do things.

 

indigoxela's picture

Wed, May 26, 2021 - 11:18pm by indigoxela

Hi brad.bulger,

I don't think, anything changed in Backdrop regarding that.

But if it worked before for you on that computer, but stopped working recently... Did anything change on that computer? Some software updates or setup changes?

There's a core issue for the symlink problem. As you were the one, who provided the pull request, you maybe had that patch applied on your install - that's why it worked, but a recent core update reverted that change...

Thu, May 27, 2021 - 8:33am by brad.bulger

That's why I'm confused. My patch was in place. But the diversion to the authorize.php page happens before the download takes place, doesn't it? The whole point of it being that it thinks it can't download directly onto the server? I think I've traced this down before.

I've already done the core update directly from a downloaded tarball because it's a security release. Once core is updated and patches applied, contrib module updates work fine.

I have a vague memory of someone having plans to get rid of authorize.php altogether, is that happening?

indigoxela's picture

Thu, May 27, 2021 - 8:46am by indigoxela

That's why I'm confused. My patch was in place.

What the... I can understand your confusion.

But unfortunately I can't recommend anything for a solution, as I don't know your system. And even if I did - it seems tricky.

Whenever authorize.php pops up, there's something fishy with file system permissions, or the setup is at least a bit uncommon. I think, that's why people want to get rid of it altogether. However, I don't think this will happen before Backdrop 2.x.

Tue, Jan 14, 2025 - 12:15am by Graham Leach

If an insecure configuration is required for the Update manager to work properly, it should clearly state that.  IMHO it should either (A) Work or (B) Refuse to proceed if the setup is not compatible.  What should be discouraged is big red error messages, which are alarming.

If CLI magic is going to always be required then the Administrative GUI should say as much, and offer a link to training or even a video explaining what needs to be done.

If the Update manager is so brittle, it should be "hardened" so it does extensive ownership/permissions checking on the configuration of the site prior to attempting an upgrade.

The Update manager offers SSH, once the system has been suitably configured on the CLI (extensive work).  That functionality should either work, or it should be removed.  

Currently, there is no path forward for me as I have a NO FTP policy on the server in question, which gets hammered with 1000's of hack attempts per day, so I have decided to reduce my attack surface accordingly.

I will work on this problem and see what I can come up with.
 

Tue, Jan 14, 2025 - 12:29am by Graham Leach

Update - this is a long-standing issue in the Drupal realm, with extensive discussion:

https://drupal.stackexchange.com/questions/257926/enable-sftp-for-upgrades

The upshot is that, for an upgrade to occur, the web-server must be given write privileges to the codebase which is a serious security risk and not something that I would do.  Some very unsafe advice is being passed around.

Also, it appears that spurious error messages may be generated indicating connection issues when the matter at hand is actually file ownership/permisssions issues.

I am still looking into this situation.
 

Sat, Jan 18, 2025 - 3:14pm by NumerousHats

Just ran into this issue while upgrading modules on a site I'm setting up.

I'm confused.

  • Does this mean that /modules needs to be owned by www-data? (Not just that www-data has write permission)
  • Why am I able to install a module from the UI, but I can't update it? That just seems bizarre.

Sat, Jan 18, 2025 - 5:40pm by Graham Leach

Hello @NumerousHats,

I agree with your assessment.  I am looking into this situation right now.

The root of this problem is that Backdrop CMS is implemented on top of a "stack" of systems with their own security context and access permissions, coupled with the fact that these systems have been abused by hackers in the past, so things have become increasingly locked down over time.  Another factor is the code that the update routine(s) were written in must conform to ever-stricter requirements in the pursuit of runtime safety.  They were written in a gentler time.  The latest requirements from PHP 8, the base language that Backdrop CMS is implemented in, may generate runtime warnings if you are not implemented on the latest release of Backdrop CMS (1.30).

At the moment, for things to work, the functionality you want requires a specific setup.  If your installation deviates from that setup things will not work, which can be puzzling and frustrating.  Thing WILL work if you can match your setup to the reference installation, which is either not well documented or hard to find, otherwise you would not have resorted to posting your question here.  

Bringing your system into conformance with the reference system requires actions that cannot (currently) be accomplished via the GUI, so it's good that you have privileged access to your file system, because that is what is needed.

The workaround, for now, is to enable your web server process (typically this is www-data, but not always) to be able to write to sub-directories of the Backdrop CMS installation other than would be allowed normally.  

Normally, the /files subdirectory is the only one where www-data is allowed to manipulate the file system.  For updates to occur, other directories might need to be manipulated by www-data.  Depending on the nature of what is being updated, that might include /modules, /core, /themes and perhaps other locations as well.  It really depends on what is being updated.

It is important to note that performing an update may result in changes to the schema of the underlying Backdrop CMS database, so a copy/paste of files is not recommended.  What is recommended is a temporary loosening of file system security for the duration of the update (and, I feel, no longer).

Longer term, what I am attempting to do is resolve this situation by expanding the reference system to include SSH-enabled updates by default, coupled with an updated ownership/permissions strategy that would enable updates in a transparent manner more in keeping with what might today be considered a prudent security posture.  

So, for now:

Loosen -> Update -> Tighten

Hopefully soon:

Update

As for the "loosening" script:  There are others who are more in touch with that workflow than I.  They might be able to provide you with the command-line  instructions or batch scripts they use to accomplish the Loosen <-> Tighten procedure.  

Please note that the specifics of the process depends on how your site structure, ownership and permissions were implemented, so a certain amount of customization will likely be needed to match your exact situation.  

I can provide background reading on this matter if you like - I wrote a book chapter on this topic in 1998 while working for Caldera Systems, the producers of Caldera OpenLinux.  The title of the book was "Caldera OpenLinux Administrators Workbook".  It was given away to participants of their training program, conducted in Provo, Utah, which I originally attended as a student.  Caldera hired me as an instructor and staff writer after I pointed out significant shortcomings in their training, and training materials.  

Please forgive me if this seems like TMI, I just want you to know a bit about who you are dealing with, to help develop trust.

If no scripts are forthcoming, I would be happy to help you further, just reach out.

 

Sat, Jan 18, 2025 - 4:58pm by NumerousHats

I believe my permissions are set up correctly, as I can install modules from the UI. But for whatever reason, I can't update them.

I will, however, double check against that documentation.

themetman's picture

Tue, Jan 21, 2025 - 1:20am by themetman

@NumerousHats How about using Bee from the command line to upgrade core and modules.

I have some scripts using a modified version of bee in my repos on github if they are any help.

I actually use these with Ansible for multiple site upgrades.