getting  infront of json api call

After i migrated a server from apache to docker nginx, i notice that all the request coming from this nginx setup has a  infront that is not visible to the browser but when you do a script call, your script will complain about invalid json format.

What is 

Well, first of all, this characters that we can't see is actually UTF-8 BOM or Byte order mark with the byte sequence 0xEF,0xBB,0xBF at the front of the file.

What to do

Lucky, There is 2 solutions for you. You can either change all the file format to UTF-8 and i meant all of it in the folder since any file included with UTF-8 BOM will cause this problem. Hence, all the file will need to be convert to UTF-8 or remove the byte sequence 0xEF,0xBB,0xBF

You can use the following script to run recursively on your root folder which will convert all the file from utf8 BOM to utf8.

find . -type f -exec sed -i.bak -e '1s/^\xEF\xBB\xBF//' {} \; -exec rm '{}.bak' \;

Personally, this works best but you'll need to work a little bit before its completed depending on how big your folder is.

The other solution is to remove BOM on the receiving end. so once you've grab your api content, remove the BOM with the following script,

        function remove_utf8_bom($text)
                $bom = pack('H*','EFBBBF');
                $text = preg_replace("/^$bom/", '', $text);
                return $text;

or you can just do this

                $raw_body = str_replace("\xEF\xBB\xBF",'',$raw_body);

both works pretty much the same. Then you should be able to parse your json_decode normally.

For more details file replacement you can visit muzso blog. Hope it helps!


Leave a Comment

6 − 4 =