how to get the cookies from a php curl into a variable
So some guy at some other company thought it would be awesome if instead of using soap or xml-rpc or rest or any other reasonable communication protocol he just embedded all of his response as cookies in the header. I need to pull these cookies out as hopefully an array from this curl response. If I have to waste a bunch of my life writing a parser for this I will be very unhappy. Does anyone know how this can simply be done, preferably without writing anything to a file? I will be very grateful if anyone can help me out with this.
8 Answers 8
$ch = curl_init('http://www.google.com/'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // get headers too with this line curl_setopt($ch, CURLOPT_HEADER, 1); $result = curl_exec($ch); // get cookie // multi-cookie variant contributed by @Combuster in comments preg_match_all('/^Set-Cookie:\s*([^;]*)/mi', $result, $matches); $cookies = array(); foreach($matches[1] as $item) < parse_str($item, $cookie); $cookies = array_merge($cookies, $cookie); >var_dump($cookies); Unfortunately I have a feeling that this is the right answer. I think its ridiculous that curl can’t just hand me a mapped array though.
I’ll give it to you but the preg_match was wrong. I didn’t just want the session, tho I understand why you would think that. But the genius who made their system is loading the cookie with an entire response map like with a get or post. Shit like this: Set-Cookie: price=1 Set-Cookie: status=accept I needed a preg_match_all with ‘/^Set-Cookie: (.*?)=(.*?)$/sm’
@thirsty93 Curl cant hand you a mapped array. But shows you a way to save it curl_setopt($ch, CURLOPT_HEADERFUNCTION, ‘callback_SaveHeaders’);
Depending on the cookie structure returned, the last line might need to be modified to something like parse_str($m[1], $cookies) , which will stuff the cookies into an associative array in the $cookies variable.
For the combined fixes that grabs more than one cookie: preg_match_all(‘/^Set-Cookie:\s*([^;]*)/mi’, $result, $matches); $cookies = array(); foreach($matches[1] as $item) < parse_str($item, $cookie); $cookies = array_merge($cookies, $cookie); >
Although this question is quite old, and the accepted response is valid, I find it a bit unconfortable because the content of the HTTP response (HTML, XML, JSON, binary or whatever) becomes mixed with the headers.
I’ve found a different alternative. CURL provides an option ( CURLOPT_HEADERFUNCTION ) to set a callback that will be called for each response header line. The function will receive the curl object and a string with the header line.
You can use a code like this (adapted from TML response):
$cookies = Array(); $ch = curl_init('http://www.google.com/'); // Ask for the callback. curl_setopt($ch, CURLOPT_HEADERFUNCTION, "curlResponseHeaderCallback"); $result = curl_exec($ch); var_dump($cookies); function curlResponseHeaderCallback($ch, $headerLine) < global $cookies; if (preg_match('/^Set-Cookie:\s*([^;]*)/mi', $headerLine, $cookie) == 1) $cookies[] = $cookie; return strlen($headerLine); // Needed by curl >This solution has the drawback of using a global variable, but I guess this is not an issue for short scripts. And you can always use static methods and attributes if curl is being wrapped into a class.
Instead of a global, you can use a closure holding a reference to $cookies . $curlResponseHeaderCallback = function ($ch, $headerLine) use (&$cookies) < then curl_setopt($ch, CURLOPT_HEADERFUNCTION, $curlResponseHeaderCallback); .
What happens if you have this all in a class? How do you reference the class function $class->curlResponseHeaderCallback() ? Or do you just have curlResponseHeaderCallback outside of the class?
This does it without regexps, but requires the PECL HTTP extension.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HEADER, 1); $result = curl_exec($ch); curl_close($ch); $headers = http_parse_headers($result); $cookobjs = Array(); foreach($headers AS $k => $v) < if (strtolower($k)=="set-cookie")< foreach($v AS $k2 =>$v2) < $cookobjs[] = http_parse_cookie($v2); >> > $cookies = Array(); foreach($cookobjs AS $row)< $cookies[] = $row->cookies; > $tmp = Array(); // sort k=>v format foreach($cookies AS $v) < foreach ($v AS $k1 =>$v1) < $tmp[$k1]=$v1; >> $cookies = $tmp; print_r($cookies); If you use CURLOPT_COOKIE_FILE and CURLOPT_COOKIE_JAR curl will read/write the cookies from/to a file. You can, after curl is done with it, read and/or modify it however you want.
libcurl also provides CURLOPT_COOKIELIST which extracts all known cookies. All you need is to make sure the PHP/CURL binding can use it.
someone here may find it useful. hhb_curl_exec2 works pretty much like curl_exec, but arg3 is an array which will be populated with the returned http headers (numeric index), and arg4 is an array which will be populated with the returned cookies ($cookies[«expires»]=>»Fri, 06-May-2016 05:58:51 GMT»), and arg5 will be populated with. info about the raw request made by curl.
the downside is that it requires CURLOPT_RETURNTRANSFER to be on, else it error out, and that it will overwrite CURLOPT_STDERR and CURLOPT_VERBOSE, if you were already using them for something else.. (i might fix this later)
function hhb_curl_exec2($ch, $url, &$returnHeaders = array(), &$returnCookies = array(), &$verboseDebugInfo = "") < $returnHeaders = array(); $returnCookies = array(); $verboseDebugInfo = ""; if (!is_resource($ch) || get_resource_type($ch) !== 'curl') < throw new InvalidArgumentException('$ch must be a curl handle!'); >if (!is_string($url)) < throw new InvalidArgumentException('$url must be a string!'); >$verbosefileh = tmpfile(); $verbosefile = stream_get_meta_data($verbosefileh); $verbosefile = $verbosefile['uri']; curl_setopt($ch, CURLOPT_VERBOSE, 1); curl_setopt($ch, CURLOPT_STDERR, $verbosefileh); curl_setopt($ch, CURLOPT_HEADER, 1); $html = hhb_curl_exec($ch, $url); $verboseDebugInfo = file_get_contents($verbosefile); curl_setopt($ch, CURLOPT_STDERR, NULL); fclose($verbosefileh); unset($verbosefile, $verbosefileh); $headers = array(); $crlf = "\x0d\x0a"; $thepos = strpos($html, $crlf . $crlf, 0); $headersString = substr($html, 0, $thepos); $headerArr = explode($crlf, $headersString); $returnHeaders = $headerArr; unset($headersString, $headerArr); $htmlBody = substr($html, $thepos + 4); //should work on utf8/ascii headers. utf32? not so sure.. unset($html); //I REALLY HOPE THERE EXIST A BETTER WAY TO GET COOKIES.. good grief this looks ugly.. //at least it's tested and seems to work perfectly. $grabCookieName = function($str) < $ret = ""; $i = 0; for ($i = 0; $i < strlen($str); ++$i) < if ($str[$i] === ' ') < continue; >if ($str[$i] === '=') < break; >$ret .= $str[$i]; > return urldecode($ret); >; foreach ($returnHeaders as $header) < //Set-Cookie: crlfcoookielol=crlf+is%0D%0A+and+newline+is+%0D%0A+and+semicolon+is%3B+and+not+sure+what+else /*Set-Cookie:ci_spill=a%3A4%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%22305d3d67b8016ca9661c3b032d4319df%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A14%3A%2285.164.158.128%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A109%3A%22Mozilla%2F5.0+%28Windows+NT+6.1%3B+WOW64%29+AppleWebKit%2F537.36+%28KHTML%2C+like+Gecko%29+Chrome%2F43.0.2357.132+Safari%2F537.36%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1436874639%3B%7Dcab1dd09f4eca466660e8a767856d013; expires=Tue, 14-Jul-2015 13:50:39 GMT; path=/ Set-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT; //Cookie names cannot contain any of the following '=,; \t\r\n\013\014' // */ if (stripos($header, "Set-Cookie:") !== 0) < continue; /**/ >$header = trim(substr($header, strlen("Set-Cookie:"))); while (strlen($header) > 0) < $cookiename = $grabCookieName($header); $returnCookies[$cookiename] = ''; $header = substr($header, strlen($cookiename) + 1); //also remove the = if (strlen($header) < 1) < break; >; $thepos = strpos($header, ';'); if ($thepos === false) < //last cookie in this Set-Cookie. $returnCookies[$cookiename] = urldecode($header); break; >$returnCookies[$cookiename] = urldecode(substr($header, 0, $thepos)); $header = trim(substr($header, $thepos + 1)); //also remove the ; > > unset($header, $cookiename, $thepos); return $htmlBody; > function hhb_curl_exec($ch, $url) < static $hhb_curl_domainCache = ""; //$hhb_curl_domainCache=&$this->hhb_curl_domainCache; //$ch=&$this->curlh; if (!is_resource($ch) || get_resource_type($ch) !== 'curl') < throw new InvalidArgumentException('$ch must be a curl handle!'); >if (!is_string($url)) < throw new InvalidArgumentException('$url must be a string!'); >$tmpvar = ""; if (parse_url($url, PHP_URL_HOST) === null) < if (substr($url, 0, 1) !== '/') < $url = $hhb_curl_domainCache . '/' . $url; >else < $url = $hhb_curl_domainCache . $url; >> ; curl_setopt($ch, CURLOPT_URL, $url); $html = curl_exec($ch); if (curl_errno($ch)) < throw new Exception('Curl error (curl_errno=' . curl_errno($ch) . ') on url ' . var_export($url, true) . ': ' . curl_error($ch)); // echo 'Curl error: ' . curl_error($ch); >if ($html === '' && 203 != ($tmpvar = curl_getinfo($ch, CURLINFO_HTTP_CODE)) /*203 is "success, but no output"..*/ ) < throw new Exception('Curl returned nothing for ' . var_export($url, true) . ' but HTTP_RESPONSE_CODE was ' . var_export($tmpvar, true)); >; //remember that curl (usually) auto-follows the "Location: " http redirects.. $hhb_curl_domainCache = parse_url(curl_getinfo($ch, CURLINFO_EFFECTIVE_URL), PHP_URL_HOST); return $html; > How to get Cookie in request header while curl php
I am trying to check login information for a website using curl php. The Request header looks like this:
GET /loginpage.cgi HTTP/1.1 Host: test.com Connection: keep-alive pragma: no-cache Cache-Control: no-cache, no-store, max-age=0, must-revalidate User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36 Expires: Fri, 01 Jan 1990 00:00:00 GMT Accept: */* Referer: https://test.com/login.cgi Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.8 Cookie: QSESSIONID=**c0da73be1917324c157e5c45b1bccd4f** The QSESSIONID value changes every time, I need this value to be passes in the header in order to make a curl request from php. My php code looks like:
Now when I run this. When I get the session id from the dev tools request header and change it in my header then only I can log in, because the value changes every time some one login. So my question is, is their a way I can get that value from the request header so that i can append that value in my header in php code. Or any other way that you can suggest, I am open for suggestions. If i remove this information from php code the request fails, if
1 Answer 1
That looks like a session ID (the name kinda gives it away), these are generated on your first connection. You’re supposed to store this and send this as a cookie in subsequent requests so the server can keep track of your session.
If you dont send this cookie header you will get a random one with every request.
This code is written in a way to attempt to teach you how cookies are handled. There are a lot of assumptions about what you are trying to do, but this gives you a basic understanding of how to parse/handle headers and the session ID.
I’ve avoided using a cookiejar (which is much easier and cleaner to code) because it automatically does all of this for you, i recommend you look into them once you learn how session ID’s work.
loggedIn = ($response == 'true'); curl_close($ch); return $this->loggedIn; > function getHeader($header) < return array_key_exists($header, $this->headers) ? $this->headers[$header] : null; > function getCookie($cookie) < return array_key_exists($cookie, $this->cookies) ? $this->cookies[$cookie] : null; > function parseHeaders($ch, $header) < if (stristr($header, 'set-cookie')) < # If you can install PECL pecl_http this will work better // $this->cookies = http_parse_cookie(strstr('Set-Cookie', $header))['cookies']; # Otherwise $reserved_words = [ 'httponly', 'expire', 'path', 'expires', 'domain', 'secure' ]; preg_match("/Set-Cookie: (.*)/", $header, $cookies); foreach ($cookies as $cookie) < $cookie = explode(';', $cookie); foreach ($cookie as $cookie_part) < $cookie_part = explode('=', $cookie_part); array_walk($cookie_part, create_function('&$val', '$val = trim($val);')); if (!in_array($cookie_part[0], $reserved_words) && isset($cookie_part[1])) < $this->cookies[$cookie_part[0]] = $cookie_part[1]; > > > > else < $header_part = explode(':', $header, 2); if (isset($header_part[1])) < $this->headers[trim($header_part[0])] = trim($header_part[1]); > > > function otherInfo() < if (!$this->loggedIn) < throw new NotLoggedInException('Login first'); >$headers = []; # Populate whatever headers are mandatory $url = "https://test.com/login.cgi?key=GetOtherInfo"; $ch = curl_init(); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_COOKIE, 'QSESSIONID=' . $this->getCookie('QSESSIONID')); $result = curl_exec($ch); curl_close($ch); return $result; > function getUserInfo() < if (!$this->loggedIn) < throw new NotLoggedInException('Login first'); >$headers = []; # Populate whatever headers are mandatory $url = "https://test.com/login.cgi?key=GetLoginUserInfo"; $ch = curl_init(); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_COOKIE, 'QSESSIONID=' . $this->getCookie('QSESSIONID')); $result = curl_exec($ch); curl_close($ch); return $result; > > $username = 'user'; $password = 'pass'; $api = new MyService(); $api->login($username, $password); $info = $api->getUserInfo(); $other = $api->otherInfo();