wp_remote_request() dies silently

private function execute( $url, $request ) {
		
		$utils = Utilities::get_instance();
		$utils->log("Executing remote request..."); // See this
		
		$start_of_request = current_time( 'timestamp' );
		$response         = wp_remote_request( $url, $request );
		$end_of_request   = current_time( 'timestamp' );
		
		$current_timeout_limit = floor( ( isset( $request['timeout'] ) ? $request['timeout'] : 60 ) * 0.8 );
		$time_of_request       = ( $end_of_request - $start_of_request );
		$max_php_execution     = ( intval( ini_get( 'max_execution_time' ) ) * 0.5 );
		
		$utils->log( "Request took {$time_of_request} seconds..." ); // Not seeing this!?!
		
		if ( $current_timeout_limit >= $max_php_execution ) {
			$msg = __( 'Error: Timeout >= half of PHP Max Execution Time!', Controller::plugin_slug );
			$utils->log( $msg );
			$utils->add_message( $msg, 'error', 'backend' );
		}
		
		// Extend the timeout value (dynamically)?
		if ( $current_timeout_limit < $time_of_request && $current_timeout_limit < $max_php_execution ) {
			$utils->log( "Have to extend the timeout value ({$time_of_request}) by 50%" );
			$this->url_args['timeout'] = $time_of_request + ceil( $time_of_request * .5 );
		}
		
		return $response;
	}

I suggest you check the wp_remote_request() response for a WP_Error object. It can relay more detailed messaging about what went wrong. Check with is_wp_error(). If an error, you can use WP_Error::get_error_messages() and relay them to your logging scheme.

Hot tip: Avoid posting replies to your own topic until after you get a reply. When you post again, your topic falls off of the “no replies” list many regulars use to find those still needing help. You can edit your OP for 30 minutes if you think of something that should be added.

I suggest you check the wp_remote_request() response for a WP_Error object.

Thanks, it’s something I do already (and doesn’t give any info in the case above).

$resp = $this->execute( $url, $request ); // $resp = wp_remote_get( $url, $request );
$code = wp_remote_retrieve_response_code( $resp );
		
if ( is_wp_error( $resp ) || 200 > $code || 300 <= $code ) {
			
	$errors = $this->process_error( $resp );			
	$msg    = "Error getting info for {$user_data->ID}: {$errors->title} - {$errors->detail}";

	$utils->log( $msg );
						
	return false;
}

FWIW, I didn’t have an “edit” option after submitting, thus the suboptimal follow-up on my post.

Checking for WP_Error is not the same as checking response codes. There can be more specific information from the function call beyond response from the remote server.

I’m sorry your were unable to edit your OP. Something is not right that we’ll need to investigate.

Hi again.

Checking for WP_Error is not the same as checking response codes

You are, of course, 100% correct.

This is why the $this->process_error() call checks for both. A call returned with status 4xx is still a valid return, it’s just that the body of the return from a number of web services (mailchimp included) contains attempted helpful information. Thus so wp_remote_request() doesn’t return a \WP_Error() object in those cases and I had to check the status as well as wp_is_error().

Instead, _when_ it returns, it returns a status that isn’t one of the 2xx or 3xx statuses (non-error/redirect).

The problem I’m trying to get to the bottom of has to do with these 3 lines of code from my original follow-up:

$start_of_request = current_time( 'timestamp' );
$response         = wp_remote_request( $url, $request );
$end_of_request   = current_time( 'timestamp' );

The $response = wp_remote_request(); call will sometimes _NOT_ proceed to $end_of_request = current_time( 'timestamp' ); and the process exits completely (_NO_ code following the wp_remote_request() is executed. Ever!).

This isn’t happening with every call, only intermittently, which is the weird part.

I was thinking it may be related to Process management on the server, or a connection timeout.

However the connection timeout theory seems unlikely since no request before that runs for more than < 1 to between 1-2 seconds. A single connection – after 30+ successful ones at 1-2 seconds – suddenly taking > 1 _minute_ to complete, the current timeout I have set in the request header, seems highly unlikely. Regardless of the server environment.

Based on Mailchimp’s documentation, it’s unlikely to be due to connection throttling since I’m at most doing 1 simultaneous connection per second and their limit is 10 simultaneous connections at one time.

Thanks for explaining. It caused me to remember that you initially did say it quietly died. I’d forgotten that detail between replies, my apologies. It’s just hard to envision this dying without logging some kind of error. Even time out errors should still return an WP_Error. Without any kind of clue I’m at a loss for suggestions.

Do you know of a specific request that would reliably induce this error? As an active, not background process. You could dig into the core code and zero in on just how far code gets before failure. Knowing that would be a big clue if not directly indicating the problem.

You could try a different HTTP transport. WP uses cURL when available, otherwise a socket transport. You can force socket by renaming the cURL transport class file at /wp-includes/Requests/Transport/cURL.php

I think I’ve figured out what’s happening…

It seems I’m running into a configuration “issue” on the web server vs PHP’s max_execution_timeout.

The PHP max_execution_timeout value is 120 seconds.

However, I believe the web host is using either FcgidBusyTimeout or the Apache Timeout setting and have whichever it is configured at 60 seconds.

As a result, my background process, thinking it’s allowed to run for 2x that amount of time, is killed by the web server.

It’s “silent” because I probably cannot see the global error logs (The host seems to think I can, but most web servers I know of will log a 500 status when they terminate a process and I don’t have any 500 entries in “my” logs).

The randomness of the issue can most likely be attributed to the fact that the job is running on a shared instance so the load of the server contributes to how long everything takes (causing the # of request issued by the job to vary).

I’ll test for a couple of days before I close this topic, just in case.

Thank you for helping me figure this out!

Resolved: The FcgidBusyTimeout and/or Timeout setting for the web server will terminate the PHP process.

The termination of the process is silent (not showing up in the error logs I can see) because I’m in a shared hosting environment with limited access to Apache error logs.