Upgrade to Fedora 30 Complete!

The last couple days there has been some extended downtime on this site. That is because I’ve been working on migrating my blog from Ubuntu 16.04 to Fedora 30. I’m switching for lots of reasons. Some of the php packages I need for WordPress have been getting a bit out of date on Ubuntu 16.04 and I wanted to have the most up-to-date stable release of php without needing to add a third party repository and Fedora 30 comes with php7.3 by default which is what is recommended by the good people at wordpress dot org (https://wordpress.org/about/requirements/).

Why make the switch from Ubuntu to Fedora? The biggest reason is that these days I’m more comfortable working in Red Hat space, and I like some of the features in Fedora 30 like the new module framework (https://docs.pagure.org/modularity/) that allows multiple versions of programming languages, and databases to be installed on the same server without conflicting with each other. Also I still live under the delusion that, someday, I will port all or some of this site into containers and I want to try Podman (https://podman.io). It’s also because the database server behind this site has been running on Fedora since just a few months after the Fedora 29 release, and after a flawless upgrade from 29 to 30 I decided that I wanted to have a consistent OS layer between the web server and the database. This is not a rebuke of Ubuntu or Debian based systems. I think they are great, they make great servers, they make great desktops, I’ve just grown more comfortable with some of the RPM based distributions, and I like that Fedora seems to be able to walk the line between stability, and keeping up with some of the newest packages available.

Lesson’s learned

SELinux – check the audit log

Moving the site was fairly easy. I just archived my web root directory and copied it over to the new server, unpacked in the same directory on the new server and that was pretty much it.

However, if you have SELinux enabled on your web server, and I recommend that you do, then you will need to flip a few sebools to allow the web server to connect to the database, and be able to install themes/updates

sudo setsebool -P httpd_can_network_connect_db 1

That setting will allow you to connect to a remote database. If you are running WordPress with the database on your web server then you don’t have to worry about that one. The next one is important if you want to allow WordPress to install plugins/themes/updates

setsebool -P httpd_can_network_connect 1

I figured this out after spending way too much time trying to figure out why my site couldn’t talk to to the database server. I knew I could hit the database port after running an nmap which meant that I didn’t have any firewall issues, and I could connect to the database with WordPress credentials. I could’ve figured it out much faster had I checked the audit log

sudo sealert -a /var/log/audit/audit.log

Checking the audit log would’ve saved me a bunch of time because it basically tells you what you need to do:

*****  Plugin catchall_boolean (47.5 confidence) suggests   ******************

If you want to allow httpd to can network connect
Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean.

Do
setsebool -P httpd_can_network_connect 1

I may end up doing a whole post for how to navigate the ins and outs of SELinux related to WordPress. These aren’t the only settings you will want to change. You need to be able to make sure that apache can read and write to a few directories which selinux will block by default, at least on Fedora 30.

Lesson learned…. read the logs first.

Force Local Users and Groups with Ansible

I’m in the process of migrating a few Puppet modules over to Ansible, and in the process I’ve run into an unusual situation while creating users and groups. Here is some background. I have an application that will refuse to complete its installation unless it can see certain users and groups in the local passwd and group files. It just so happens that these same users and groups are also contained in LDAP.

Puppet has an attribue called “forcelocal” in its user and group resource that has always been able to create a local user or group in this situation, despite having a matching user or group in LDAP. So, I was a bit dissappointed to discover that the similar “local” option in both the group and the user Ansible modules did not work in the same way. From the user module docs, the “local” option has the following behavior:

“Forces the use of “local” command alternatives on platforms that implement it. This is useful in environments that use centralized authentification when you want to manipulate the local users. I.E. it uses `luseradd` instead of `useradd`.This requires that these commands exist on the targeted host, otherwise it will be a fatal error. “ (https://docs.ansible.com/ansible/latest/modules/user_module.html#user-module)

After reading that, I expected Ansible to create local users and groups regardless of whether or not that user or group was already found in LDAP. However, that is not the case. For whatever reason specifying the local option does not create a local user or group, if that user or group is already in LDAP, and is visible to your target server. Instead, Ansible will simply mark the task as complete and happily move on to the next step. Looking at the code for the module it’s using “grp” from the standard library, so it will just check the user database for the user or group, since it finds the user (albeit in LDAP) it moves on, which for my use case kinda defeats the whole purpose of the local option. I would like to see this module do a further check to see if the specified user name or group was listed in the /etc/passwd or /etc/group files, before marking success.

After a bit of head scratching, and cursing I read a few blogs, and stack exchange solutions from others who had attempted to solve this, but none of them struck me as viable for my situation. Because I’m not enough of a programmer to fix this little bug, and since I only need to run this particular playbook once, at the time a server is deployed, I chose a bit of a compromise solution.

At first, I kicked around the idea of inserting the user and group directly into the passwd, shadow, and group files, but that just didn’t seem like a clean solution to me. Plus, I assume at some point this problem will be fixed, so it seems easier to continue using the group and user modules than to rewrite the playbook at some point in the near future. So I decided to do the following: stop the sssd service (thus making the LDAP users and groups invisible to the server), add the users and groups using the Ansible module, and then restart sssd.

Here is a slimmed down version of what I ended up doing in the playbook. Keep in mind, it will only work if you are using sssd you should also make sure that your server is in a state where sssd can be stopped while these tasks are processed. In my case it’s fine because I only run this particular sequence once when the server is built.

---
- name: stop sssd
  service:
    name: sssd
    state: stopped
- name: add group
  group:
    name: localgroup
    gid: 1234
    state: present
- name: add user
  user:
    name: localuser
    uid: 1234
    group: localgroup
    state: present
- name: start sssd
  service:
    name: sssd
    state: started
    enabled: yes
 ...

Let me know if you’ve run into this, or if you have a better solution. I suspect you could make this a bit more universal by adding a test to see if the user and group have an entry in the passwd and group files before stopping sssd.

Convert a pem file into a rsa private key

When you build a server in AWS one of the last steps is to either acknowledge that you have access to an existing pem file, or to create a new one to use when authenticating to your ec2 server.

If you want to convert that file into an rsa key that you can use in an ssh config file, you can use this handy dandy openssl command string.

openssl rsa -in somefile.pem -out id_rsa

Note: you don’t have to call the output file id_rsa, you will want to make sure that you don’t overwrite an existing id_rsa file.

Copy the id_rsa file to your .ssh directory and make sure to change permissions on the id_rsa key to read only for just your user.

chmod 400 ~/.ssh/id_rsa