I'm processing a large JSON File (>1500 items). Im using JSONStreamParser to process one item at time. With each one, I am updating a post if the data has changed. The first thing I need to do is check to see if we already have that item in the database, so I am doing a lookup as shown below.
The problem is that each time the function below runs, it adds to the memory that is being used so that by the time we get through 500 items, the memory has grown by 50MB and then the whole process just quits.
Is there something I can do to release that memory? Is there a better way to do this?
function get_page_by_loclink($loclink) {
$args = array(
'post_type' => 'cm-map-location',
'meta_query' => array(
array(
'key' => 'ht_loclink',
'value' => $loclink,
'compare' => '='
)
)
);
$query = new WP_Query($args);
if ($query->have_posts()) {
$posts = $query->posts;
return $posts[0]->ID;
} else {
return false;
}
}