httplib is an libs help you to curl remote url.

you can use Get to crawl data.

import "github.com/icyxp/go-httplib"

str, err := httplib.Get("http://jiunile.com/").String()
if err != nil {
    	// error
}
fmt.Println(str)

POST data to remote url

req := httplib.Post("http://jiunile.com/")
req.Param("username","jiunile")
req.Param("password","123456")
str, err := req.String()
if err != nil {
    	// error
}
fmt.Println(str)

The default timeout is 60 seconds, function prototype:

SetTimeout(connectTimeout, readWriteTimeout time.Duration)

Example:

// GET
httplib.Get("http://jiunile.com/").SetTimeout(100 * time.Second, 30 * time.Second)

// POST
httplib.Post("http://jiunile.com/").SetTimeout(100 * time.Second, 30 * time.Second)

If you want to debug the request info, set the debug on

httplib.Get("http://jiunile.com/").Debug(true)

str, err := Get("http://jiunile.com/").SetBasicAuth("user", "passwd").String()
if err != nil {
    	// error
}
fmt.Println(str)

If request url is https, You can set the client support TSL:

httplib.SetTLSClientConfig(&tls.Config{InsecureSkipVerify: true})

More info about the tls.Config please visit http://golang.org/pkg/crypto/tls/#Config

some servers need to specify the protocol version of HTTP

httplib.Get("http://jiunile.com/").SetProtocolVersion("HTTP/1.1")

some http request need setcookie. So set it like this:

cookie := &http.Cookie{}
cookie.Name = "username"
cookie.Value  = "jiunile"
httplib.Get("http://jiunile.com/").SetCookie(cookie)

httplib support mutil file upload, use req.PostFile()

req := httplib.Post("http://jiunile.com/")
req.Param("username","jiunile")
req.PostFile("uploadfile1", "httplib.pdf")
str, err := req.String()
if err != nil {
    	// error
}
fmt.Println(str)

See godoc for further documentation and examples.

• godoc.org/github.com/icyxp/go-httplib