Cache Image failed - But file seems OK - Lighttp issue?

Support requests, bug reports, etc. go here. Dedicated servers / VDS hosting only
alvarord
Bear Rating Trainee
Bear Rating Trainee
Posts: 11
Joined: 18 Apr 2013, 08:53

Cache Image failed - But file seems OK - Lighttp issue?

Postby alvarord » 10 Jan 2015, 11:42

Hello,

I have been checking why this feed (http://listo.tumblr.com/rss) doesn't show proper images inside tttrs, I haven't noticed yet about any other image issue, and with this feed seems to happens always.

So I review a little the code and check the feed entry:

Code: Select all

ttrss> select * from ttrss_entries where link = 'http://listo.tumblr.com/post/107616572630'
+--------+---------+-----------------------------------------------+-------------------------------------------+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+------------------+----------------+---------------------+----------------------------+----------------+------------+-----------------------------------------+----------+--------+
|     id | title   | guid                                          | link                                      | updated             | content                                                                 
                                                                              | content_hash                             |   cached_content |   no_orig_date | date_entered        | date_updated               |   num_comments | comments   | plugin_data                             | author   | lang   |
|--------+---------+-----------------------------------------------+-------------------------------------------+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+------------------+----------------+---------------------+----------------------------+----------------+------------+-----------------------------------------+----------+--------|
| 109850 | Photo   | SHA1:82c465d836440174a0bad9c7e24ad3e5c590bc92 | http://listo.tumblr.com/post/107616572630 | 2015-01-09 19:30:02 | <html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><?xml version="1.0" encoding="utf-8" standalone="yes"?><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body><img src="http://40.media.tumblr.com/683127fde9b2d1b37c21d0eb072e8079/tumblr_nhxcu2BCsP1qz778zo1_1280.jpg"><br><br></body></html> | 34c9b389566f6513ec6f4708362c99cbbe1ca474 |                  |              0 | 2015-01-09 19:48:00 | 2015-01-10 06:59:06.347233 |              0 |            | af_fsckportal,af_tumblr_1280,af_unburn, |          | pt     |


Where I noticed that the image seems ok: "http://40.media.tumblr.com/683127fde9b2d1b37c21d0eb072e8079/tumblr_nhxcu2BCsP1qz778zo1_1280.jpg" and it is downloadable but in the browser when I analyze the URL it is seen: "image.php?hash=fc48b4c46355558cf4b3a2146ffd52ed2cf703b0"

I download the link with curl make the md5sum and made the same with the file stored as fc48b4c46355558cf4b3a2146ffd52ed2cf703b0.png inside cache directory and the result was the same but when I do the request to "image.php?hash=fc48b4c46355558cf4b3a2146ffd52ed2cf703b0" with firefox it is not able to show it and the md5sum with curl is different than the one in the file.

Im not using apache, I'm using lighttp and maybe the readfile is failing?

Code: Select all

                        if (function_exists('apache_get_modules') &&
                            array_search('mod_xsendfile', apache_get_modules()))
                                $xsendfile = true;

                        if ($xsendfile) {
                                header("X-Sendfile: $filename");
                                header("Content-type: application/octet-stream");
                                header('Content-Disposition: attachment; filename="' . basename($filename) . '"');
                        } else {
                                header("Content-type: image/png");
                                $stamp = gmdate("D, d M Y H:i:s", filemtime($filename)). " GMT";
                                header("Last-Modified: $stamp", true);
                                readfile($filename);
                        }



Someone using a lighttp with the same issue? Any idea to keep tracking the issue?

alvarord
Bear Rating Trainee
Bear Rating Trainee
Posts: 11
Joined: 18 Apr 2013, 08:53

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby alvarord » 10 Jan 2015, 11:54

A fast test with a readfile.php code seems to work and serve properly the png file, and the md5sum is correct.

Code: Select all

curl -v https://server.com/testreadfile.php -o /tmp/test ; md5sum /tmp/test
.......
> GET /testreadfile.php HTTP/1.1
> User-Agent: curl/7.35.0
> Host: server.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Strict-Transport-Security: max-age=63072000
< Content-Description: File Transfer
< Content-Type: application/octet-stream
< Content-Disposition: attachment; filename=577626bb01beeb1c28992463e8dc1aa1a51f11b0.png
< Expires: 0
< Cache-Control: must-revalidate
< Pragma: public
< Content-Length: 47204
< Date: Sat, 10 Jan 2015 08:55:15 GMT
* Server lighttpd is not blacklisted
< Server: lighttpd
<
{ [data not shown]
100 47204  100 47204    0     0   732k      0 --:--:-- --:--:-- --:--:--  743k
* Connection #0 to host server.com left intact
007dbb0ecf45a29a366c948d7476c6a7  /tmp/test
###:~⟫ md5sum /var/www/ttrss/cache/images/577626bb01beeb1c28992463e8dc1aa1a51f11b0.png
007dbb0ecf45a29a366c948d7476c6a7  /var/www/ttrss/cache/images/577626bb01beeb1c28992463e8dc1aa1a51f11b0.png


Code: Select all

/var/www# cat testreadfile.php
<?php
$file = 'ttrss/cache/images/577626bb01beeb1c28992463e8dc1aa1a51f11b0.png';

if (file_exists($file)) {
            header('Content-Description: File Transfer');
                header('Content-Type: application/octet-stream');
                header('Content-Disposition: attachment; filename='.basename($file));
                    header('Expires: 0');
                    header('Cache-Control: must-revalidate');
                        header('Pragma: public');
                        header('Content-Length: ' . filesize($file));
                            readfile($file);
                            exit;
}
?>


I don't know what to review now.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby fox » 10 Jan 2015, 12:47

i was not aware this was lighthttpd support central

dijsil
Bear Rating Trainee
Bear Rating Trainee
Posts: 5
Joined: 20 Dec 2013, 20:45

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby dijsil » 10 Jan 2015, 12:51

I have had a similar problem in the past.
Conditions:
  • you have the image cache turned on for that feed (hence the <server>/image.php?hash=xxx url)
  • the conversion of the specific image to the png-format (which should be done by the tt-rss backend) has failed but the file still gets renamed

In this case the image.php url sends out the image with the wrong header (saying it is a png file, while is actually sends out a jpg/... file).
This sometimes causes images not to be shown in the browser.

Proposed fix:
make sure that image.php sends out the correct header for each file, since the conversion is not always reliable.

edit: this is not related to lighthttpd, also happens on apache

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby fox » 10 Jan 2015, 13:05

nothing actually gets converted to png, stop posting stupid shit

it just gets served with image/png because it literally doesn't matter to any sensible browser which would just figure out the actual type on its own and figuring out actual mimetype before sending is a waste of cpu time

dijsil
Bear Rating Trainee
Bear Rating Trainee
Posts: 5
Joined: 20 Dec 2013, 20:45

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby dijsil » 10 Jan 2015, 14:10

so shifting cpu-usage from a server (which usually has plenty) to the client (which is forced to determine what you are actually serving despite all the wrong info you send out) is not a waste of cpu time?
The cpu-usage is there in both cases, the question that remains: do you want it on the server, or on the client.

Since the server usually has more cpu-power, I suggest the server-side to be fixed.
(this would also reduce cpu-load on the tablets which use your android-app, and there cpu-power is a premium cost)

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby fox » 10 Jan 2015, 14:27

your opinions are bad and dumb, hth

e: especially this gem:

>Since the server usually has more cpu-power, I suggest the server-side to be fixed.

e2: also bonus lols at you seriously thinking that actually determining image type before decoding adds any significant cpu overhead to the client you dumb fuck, stop posting here

alvarord
Bear Rating Trainee
Bear Rating Trainee
Posts: 11
Joined: 18 Apr 2013, 08:53

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby alvarord » 10 Jan 2015, 15:12

I think that fox is right and the issue is not in the tinytinyrss so I will try to figure it out why the answer is different, because it doesn't make too much sense. And the readfile function seems to works properly.

If not I will deactivate the cache.

alvarord
Bear Rating Trainee
Bear Rating Trainee
Posts: 11
Joined: 18 Apr 2013, 08:53

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby alvarord » 10 Jan 2015, 15:14

I think that fox is right and the issue is not in the tinytinyrss so I will try to figure it out why the server response is different, because it doesn't make too much sense. And the readfile function seems to works properly.

If not I will deactivate the cache.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby fox » 10 Jan 2015, 15:24

maybe it's some transparent gzip / ob_gzhandler shit misfiring?

alvarord
Bear Rating Trainee
Bear Rating Trainee
Posts: 11
Joined: 18 Apr 2013, 08:53

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby alvarord » 10 Jan 2015, 20:14

I will try to check that.

Thanks for the ideas.

alvarord
Bear Rating Trainee
Bear Rating Trainee
Posts: 11
Joined: 18 Apr 2013, 08:53

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby alvarord » 11 Jan 2015, 12:10

What I have found:

I have seen that the failure is produced because with CURL I receive always one byte more in the first position: exactly a "\x0a". I remove it from the fetched file and same md5sum as the file in cache.

Cache image for a specificfeed fail BUT cache image from starred item doesnt. So requests behave different in the same server:
ttrss/image.php?hash=$hash - add one byte '\x0a', I'm really newbie with PHP and I don't know where it can came from.
ttrss/backend.php?op=pluginhandler&plugin=cache_starred_images&method=image&hash=$hash - return the proper file. Same md5sum with curl

I checked the code in plugins/cache_starred_images/init.php and it look similar to the one in image.php but there is no ob_end_clean in image.php I tried to add it but the issue persists. I have been playing with ob_gzhandler without success yet.

With CURL when I do the request to image.php, no authentication is needed but to download the cached starred image I need to be authenticated.

I will keep try to check why through backend it works, but I suppose I will need to read much more code.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby fox » 11 Jan 2015, 12:27

maybe you added some blank lines between <? ?> tags for example in config.php

alvarord
Bear Rating Trainee
Bear Rating Trainee
Posts: 11
Joined: 18 Apr 2013, 08:53

Re: Cache Image failed - But file seems OK - Lighttp issue?

Postby alvarord » 11 Jan 2015, 19:21

Thanks.

That solve the issue:

Code: Select all

# diff -u config.php config.php.issuecache
--- config.php  2015-01-11 17:04:03.823721000 +0100
+++ config.php.issuecache       2015-01-11 12:09:34.283721000 +0100
@@ -218,3 +218,4 @@
 
        // vim:ft=php
 ?>
+


Return to “Support”

Who is online

Users browsing this forum: Bing [Bot] and 9 guests