0

Actually I delevop a plugin that uses several curl requests to get data from an external server. I've got the following code running and is working fine. The post and meta gets in But if I running the import_images() function for insert each post and multiple image I run into 504 Gateway Timeout and the images duplicates all images. My current PHP VERSION: 7.4, PHP Memory Limit: 512 MB

/**
 * Get all vehicles.
 *
 * @return array
 */
public function request_get_vehicles() {
    $data = http_build_query(
        array(
            'api_key'             => $this->api_key,
            'get'                 => self::GET_LISTE,
            'max_autos_pro_seite' => self::MAX_AUTOS_PRO_SEITE,
        )
    );

    $curl = curl_init();

    curl_setopt( $curl, CURLOPT_URL, $this->api_url() );
    curl_setopt( $curl, CURLOPT_POST, true );
    curl_setopt( $curl, CURLOPT_POSTFIELDS, $data );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1 );

    $buffer = curl_exec( $curl );
    curl_close( $curl );

    $data = json_decode( $buffer, true );

    return $data;
}

/**
 * Undocumented function
 *
 * @param int $id Vehicle ID.
 * @return array Return image urls.
 */
public function request_get_details( $id ) {
    $data = http_build_query(
        array(
            'api_key'             => $this->api_key,
            'get'                 => self::GET_DETAIL,
            'fahrzeug_id'         => $id,
            'max_autos_pro_seite' => self::MAX_AUTOS_PRO_SEITE,
        )
    );

    $curl = curl_init();

    curl_setopt( $curl, CURLOPT_URL, $this->api_url() );
    curl_setopt( $curl, CURLOPT_POST, true );
    curl_setopt( $curl, CURLOPT_POSTFIELDS, $data );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1 );

    $buffer = curl_exec( $curl );
    curl_close( $curl );

    $data = json_decode( $buffer, true );

    return $data;
}

/**
 * Save vehicles and images in database.
 *
 * @return int The post ID on success.
 */
public function wpmc_save() {
    global $wpdb;

    $vehicles = $this->request_get_vehicles();

    if ( empty( $vehicles ) ) {
        return false;
    }

    foreach ( $vehicles['autos_liste'] as $vehicle_id => $value ) {
        $args = array(
            'post_type'    => 'fahrzeuge',
            'post_title'   => $value['marke'] . ' ' . $value['modell'] . ' ' . $value['kopfzeile'],
            'post_status'  => 'publish',
            'post_content' => ! empty( $value['bemerkung_b2b'] ) ? $value['bemerkung_b2b'] : $value['bemerkung_b2c'],
        );

        $meta_args = array(
            'meta_key'   => 'fahrzeug_id',
            'meta_value' => $value['fahrzeug_id'],
        );

        $posts = $wpdb->get_results(
            $wpdb->prepare(
                "SELECT * FROM {$wpdb->postmeta}
                WHERE meta_key = 'fahrzeug_id' AND  meta_value = %d
                LIMIT 1",
                $value['fahrzeug_id']
            )
        );

        if ( empty( $posts ) ) {
            $post_id = wp_insert_post( $args );
        }

        // Set meta data.
        $post_meta = $this->set_metadata( $value );

        if ( ! empty( $post_id ) ) {
            // Set meta data.
            foreach ( $post_meta as $meta_key => $meta_value ) {
                // Get the current meta value.
                $current_value = get_post_meta( $post_id, $meta_key, true );

                if ( $meta_value && '' === $current_value ) {
                    add_post_meta( $post_id, $meta_key, $meta_value );
                } elseif ( $meta_value && $meta_value !== $current_value ) {
                    update_post_meta( $post_id, $meta_key, $meta_value );
                } elseif ( '' === $meta_value && $current_value ) {
                    delete_post_meta( $post_id, $meta_key, $current_value );
                }
            }
        }

        $images = $this->request_get_details( $value['fahrzeug_id'] );

        foreach ( $images['bilder'] as $key => $value ) {
            $this->import_images( $value['url'], $post_id );
        }
    }
}

/**
 * Load images to the post.
 *
 * @param  array $url  The URL of the image to download.
 * @param  int   $post_id The post ID the media is to be associated with.
 * @return int   Image ID on success.
 */
private function import_images( $url, $post_id ) {
    // Include required files.
    require_once ABSPATH . '/wp-admin/includes/file.php';
    require_once ABSPATH . '/wp-admin/includes/media.php';
    require_once ABSPATH . '/wp-admin/includes/image.php';

    $tmp = download_url( $url );

    // Set variables for storage, fix file filename for query strings.
    preg_match( '/[^\\?]+\\.(jpe?g|jpe|gif|png)\\b/i', $url, $matches );

    $file_array['name']     = sanitize_file_name( basename( $matches[0] ) );
    $file_array['tmp_name'] = $tmp;

    if ( is_wp_error( $tmp ) ) {
        @unlink( $file_array['tmp_name'] );
        return false;
    }

    // Saves it as an attachment.
    $attachment_id = media_handle_sideload( $file_array, $post_id, null );

    if ( is_wp_error( $attachment_id ) ) {
        @unlink( $file_array['tmp_name'] );
        return false;
    }

    return $attachment_id;
}

When importing, a total of 8 different image sizes are created. Is it possible to prevent this?

enter image description here

To test my function I actually hook into 'wp' to call the function! I hope someone can help me to fix and speed up the import error.

Thanks in advance!

2
  • even if you eliminate the intermediate image size generation that may increase the number you can process before a timeout occurs but it does not solve the problem. You should separate out the two tasks of fetching and figuring out what to download, vs actually downloading. Doing all of it in a single request doesn't scale and puts a hard upper limit on how much you can do
    – Tom J Nowell
    Mar 30, 2022 at 16:09
  • Thank you very much for your feedback. Could you give me an example how you mean that? Mar 31, 2022 at 6:58

1 Answer 1

0

Is there anybody can help Figuren me out to seperate the two task and get this running? The Problem is that I need each vehicle id to get the details and query the associated images URLS.

6
  • I am unable to understand what you are trying to say.
    – Mort 1305
    Apr 2, 2022 at 17:46
  • In the first function request_get_vehicles() I get all listed vehicles and there unique ID. With that ID I can all needed metadata including the required image URLs that I want to import. So I hope that anybody can help me to seperate the second function request_get_details from the loop. So i think that's the problem I run through the timeout error! Apr 2, 2022 at 18:01
  • First, let's start with how this whole thing gets going. It looks like these are all functions of some kind of class. So, something is calling the ->wpmc_save() function, right? And, that call is what starts the process?
    – Mort 1305
    Apr 2, 2022 at 23:58
  • 1
    That's right, these are all functions of a class. The final wpmc_save() function is called for testing through the 'wp' action hook Apr 3, 2022 at 8:10
  • Here's my trace. When ->wpmc_save() is executed, a list is obtained from ->request_get_vehicles(). That list is iterated over to: create a post with metadata obtained from ->set_metadata(); and URL's obtained from ->request_get_details() are passed to ->import_images(). The ->request_vehicles() and ->request_get_details() methods do nothing outside of a cURL execution, and the ->import_images() method does nothing outside of creating an attachment from a URL. No code was provided for ->import_images() or any detail on what is calling ->wpmc_save().
    – Mort 1305
    Apr 3, 2022 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.