Update: I've confirmed the problem (that scheduled posts are based on UTC time), now just need advice and background to determine best solution. Questions:
1) how WP determines UTC time before calculating local timezone offset (i.e. does it use server OS time or some other source--can't find in codex/googling)
..which would help me answer..
2) is changing server time from UTC to local PST the only/best way to make scheduled posts use the same timezone as WP-time? and
2a) are there any potential negative implications to changing server time from UTC to local?
3) what's the best way to change server time in an accurate way--when I set a clock, it doesn't have to be exactly right, but I'd be concerned about manually setting the time on a server.
Problem: First, WP started missing publication of scheduled posts. This seems to be a widely-experienced issue, and I fixed it by using the "missed schedule" plugin. This is probably, but not necessarily, irrelevant background.
Soon after a move to a new server, WP started publishing scheduled posts early, which to me, can only point to misconfigured time/date settings somewhere in our stack.
Stack: LNMP, one nginx box, one db/memcache/php box (though I didn't, and couldn't, build it)
As far as I know, there are 4 places in our stack where time is set/relied upon: in WordPress General Settings (set to PST, but also correctly displays UTC server time), for PHP5 in php.ini (was set to central time where the new server is, but I updated to PST and restarted php-fpm once I realized PHP5 has a timezone setting, decreasing the discrepancy by 2 hours), and each box's OS (both are set to UTC time, which I have been assuming is ok because WP recognizes it as such as can still set its 'own' time).
Yet, I'm currently watching something occur that shouldn't be possible. A post was set to publish at 9:30am pst, but published at 1:30am pst instead (9:30am utc). This caused the "published X ago" relative timestamp on the post to count backwards from 8 hours, and then start counting up again at 9:30am pst. To me, this means that while the post scheduler (seemingly) should be looking to the (correctly configured) local WP time (as the frontend is), it's instead using UTC (server) time.
A. re-set both boxes' OS, plus PHP, plus all WP blogs to only use utc time, and instruct all editorial to live in UTC.
B. re-set both boxes' OS to PST to match PHP and WP. It seems like this is probably the solution (since the post published on server/UTC time), but I'm a *nix n00b and just want to make sure that I should use 'date -s' and just try to be as accurate as possible, or if it would be smarter just to leave a host support ticket asking them to sync both servers' times to PST? (or if there's a better way to be precise about it)
What I don't get:
A. Why does the post scheduler disregards "WP time" to make its calculation of when to publish? (update: I think the answer is here: http://codex.wordpress.org/Function_Reference/wp_schedule_event - it parses time via unix timestamp directly, rather than through wp?)
B. Why even have WP settings separating UTC time from local time, if all functions don't calculate time from the same source?
Why this is an issue:
We have 15 sites, with primary site users in different timezones. Ideally, they could set WP "local time" per blog, and have all post scheduling and functions rely on that specified time. If scheduling is based on server time, this seems impossible.