yum / yum-complete-transaction spawns 3 more yum processes

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • DigiState
    Junior Member
    Forum Explorer
    • Feb 2024
    • 20

    #1

    yum / yum-complete-transaction spawns 3 more yum processes

    This has been happening on multiple servers for quite awhile now. Whenever we run updates all the servers that run CloudLinux spawn yum multiple times, causing yum to wait for itself to finish.
    ps aux | grep tee shows 2 tee sessions to the same log file.

    Code:
    # ps aux | grep tee
    root       76662  0.0  0.0   4364   360 pts/0    S+   11:31   0:00 tee -ia /var/log/directadmin/custombuild.1708943469.76592.cmV3cml0ZV9jb25mcwA.log
    root       76664  0.0  0.0   4364   360 pts/0    S+   11:31   0:00 tee -ia /var/log/directadmin/custombuild.1708943469.76592.cmV3cml0ZV9jb25mcwA.log
    root       81972  0.0  0.0 110804   892 pts/1    S+   11:47   0:00 grep --color=auto tee
    ​
    and the log file shows

    Code:
    Another app is currently holding the yum lock; waiting for it to exit...
      The other application is: yum-complete-tr
        Memory : 215 M RSS (841 MB VSZ)
        Started: Mon Feb 26 11:28:48 2024 - 17:26 ago
        State  : Sleeping, pid: 75671
    Another app is currently holding the yum lock; waiting for it to exit...
      The other application is: yum-complete-tr
        Memory : 215 M RSS (841 MB VSZ)
        Started: Mon Feb 26 11:28:48 2024 - 17:28 ago
        State  : Sleeping, pid: 75671
    
    ​
    processlist:
    Code:
    \_ /usr/bin/python -tt /sbin/yum-complete-transaction
        \_ /bin/sh /var/tmp/rpm-tmp.0CeQAq 5
             \_ /usr/local/directadmin/custombuild/build rewrite_confs
                  \_ custombuild -c source /proc/self/fd/3 build rewrite_confs
                      \_ custombuild -c source /proc/self/fd/3 build rewrite_confs
                      |   \_ tee -ia /var/log/directadmin/custombuild.1708943469.76592.cmV3cml0ZV9jb25mcwA.log
                      \_ custombuild -c source /proc/self/fd/3 build rewrite_confs
                      |   \_ tee -ia /var/log/directadmin/custombuild.1708943469.76592.cmV3cml0ZV9jb25mcwA.log
                      \_ custombuild -c source /proc/self/fd/3 build rewrite_confs
                          \_ custombuild -c source /proc/self/fd/3 build rewrite_confs
                              \_ sh ./install/da_cb_install stable 1.1-72
                                  \_ /usr/bin/python /bin/yum install -y yum-utils file
    
    ​
    The waiting never ends.
  • bogdan.sh
    Administrator
    • Nov 2016
    • 1221

    #2
    Hello,

    Could you please clarify how exactly do you run updates?

    Same time, please show the content of /etc/resolv.conf file. Or try setting some good external nameservers in /etc/resolv.conf (e.g. 8.8.8.8 / 8.8.4.4) then clean cache:

    Code:
    yum clean all; yum clean metadata

    Comment

    • DigiState
      Junior Member
      Forum Explorer
      • Feb 2024
      • 20

      #3
      We use google for dns. Updates are done with

      yum -y update

      But like I said, it also happens with
      yum-complete-transaction (without options)

      Comment

      • DigiState
        Junior Member
        Forum Explorer
        • Feb 2024
        • 20

        #4
        Code:
        yum clean all; yum clean metadata
        did not make any difference.

        The wait has now gone on for 17 hours.
        Code:
        Another app is currently holding the yum lock; waiting for it to exit...
          The other application is: yum-complete-tr
            Memory : 227 M RSS (573 MB VSZ)
            Started: Mon Feb 26 15:36:16 2024 - 17:19:48 ago
            State  : Sleeping, pid: 150489
        
        ​
        /etc/resolv.conf
        Code:
        # cat /etc/resolv.conf
        # Generated by NetworkManager
        nameserver 8.8.8.8
        nameserver 8.8.4.4
        
        ​

        Comment

        • DigiState
          Junior Member
          Forum Explorer
          • Feb 2024
          • 20

          #5
          I aborted the yum-complete-transaction command I had running and am presented with the following message:

          Code:
          Another instance of custombuild (PID 152182) is holding the lock on "/usr/local/directadmin/custombuild/.custombuild.lock", exiting
          ​Which is odd because custombuild was not started, by me at least.

          Comment

          • bogdan.sh
            Administrator
            • Nov 2016
            • 1221

            #6
            What OS is that? I just tried on CloudLinux 8 and cannot reproduce the issue. Searched for similar tickets and seems we have none. Anything special in "yum history | head"?

            Comment

            • DigiState
              Junior Member
              Forum Explorer
              • Feb 2024
              • 20

              #7
              It's CloudLinux release 7.9 (Boris Yegorov)

              ​This happens on all the cloudlinux servers we have. Anything you do yum related starts custombuild.

              Comment

              • bogdan.sh
                Administrator
                • Nov 2016
                • 1221

                #8
                I am wondering, why the yum transactions become uncompleted there? Can you recall any errors from the past? Please try to clean them with a pretty safe:

                Code:
                yum-complete-transaction --cleanup-only

                Comment

                • DigiState
                  Junior Member
                  Forum Explorer
                  • Feb 2024
                  • 20

                  #9
                  That certainly got rid of the list, sure. Next round of updates is on monday, so let's see if this sticks.

                  Comment

                  • bogdan.sh
                    Administrator
                    • Nov 2016
                    • 1221

                    #10
                    Let's investigate more
                    Please try running update then perform complete-transaction to check if any unfinished transactions left?

                    Code:
                    yum update
                    yum-complete-transaction

                    Comment

                    • DigiState
                      Junior Member
                      Forum Explorer
                      • Feb 2024
                      • 20

                      #11
                      No updates since yesterday. We're going to have to wait awhile.

                      Comment

                      • DigiState
                        Junior Member
                        Forum Explorer
                        • Feb 2024
                        • 20

                        #12
                        Running updates today. There was an update for exim but it would not install. Had to remove old libiconv and build all. During the process I keep getting

                        Code:
                        Error: /etc/cagefs/cagefs.mp file contains incorrect path - /usr/local/lib/php is NOT a directory or does NOT exist​
                        whcih is weird because it does exist

                        Code:
                        ]# ls -l /usr/local/bin/php
                        lrwxrwxrwx 1 root root 26 Sep 6 11:31 /usr/local/bin/php -> /usr/local/php72/bin/php72

                        Comment

                        • bogdan.sh
                          Administrator
                          • Nov 2016
                          • 1221

                          #13
                          The cagefs.mp file should include only directories since it is used to mount them into virtual file system. While seems the /usr/local/bin/php is a symlink to binary file. The file itself is already pushed to CageFS with /etc/cagefs/conf.d/php.cfg file so please remove it from /etc/cagefs/cagefs.mp .

                          Comment

                          • DigiState
                            Junior Member
                            Forum Explorer
                            • Feb 2024
                            • 20

                            #14
                            Right. I removed the line from /etc/cagefs/cagefs.mp

                            Comment

                            • DigiState
                              Junior Member
                              Forum Explorer
                              • Feb 2024
                              • 20

                              #15
                              I'll consider this matter closed.

                              Comment

                              Working...
                              X