Resolve conflicts among master branch and develop branch (#5286)

* feature extend readthrough for cache module (#5116)

* feature 增加readthrough

* feature: add write though for cache mode (#5117)

* feature: add writethough for cache mode

* feature add singleflight cache (#5119)

* build(deps): bump go.opentelemetry.io/otel/trace from 1.8.0 to 1.11.2

Bumps [go.opentelemetry.io/otel/trace](https://github.com/open-telemetry/opentelemetry-go) from 1.8.0 to 1.11.2.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.8.0...v1.11.2)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/trace
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix 5129: must set formatter after init the logger

* remove beego.vip

* build(deps): bump actions/stale from 5 to 7

Bumps [actions/stale](https://github.com/actions/stale) from 5 to 7.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix 5079: only log msg when the channel is not closed (#5132)

* optimize test

* upgrade otel dependencies to v1.11.2

* format code

* Bloom filter cache (#5126)

* feature: add bloom filter cache

* feature upload remove all temp file

* bugfix Controller SaveToFile remove all temp file

* rft: motify BeeLogger signalChan (#5139)

* add non-block write log in asynchronous mode (#5150)

* add non-block write log in asynchronous mode

---------

Co-authored-by: chenhaokun <chenhaokun@itiger.com>

* fix the docsite URL (#5173)

* Unified gopkg.in/yaml version to v2 (#5169)

* Unified gopkg.in/yaml version to v2 and go mod tidy

* update CHANGELOG

* bugfix: protect field access with lock to avoid possible data race (#5211)

* fix some comments (#5194)

Signed-off-by: cui fliter <imcusg@gmail.com>

* build(deps): bump github.com/prometheus/client_golang (#5213)

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.14.0 to 1.15.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.14.0...v1.15.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump go.etcd.io/etcd/client/v3 from 3.5.4 to 3.5.9 (#5209)

Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.5.4 to 3.5.9.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.4...v3.5.9)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* cache: fix typo and optimize the naming

* Release 2.1.0 change log

* bugfix: beegoAppConfig String and Strings function has bug

* httplib: fix unstable test, do not use httplib.org

* chore: pkg imported more than once

* chore: fmt modify

* chore: Use github.com/go-kit/log

* chore: unnecessary use of fmt.Sprintf

* fix: golangci-lint error

* orm: refactor ORM introducing internal/models pkg

* remove adapter package

* build(deps): bump github.com/bits-and-blooms/bloom/v3

Bumps [github.com/bits-and-blooms/bloom/v3](https://github.com/bits-and-blooms/bloom) from 3.3.1 to 3.5.0.
- [Release notes](https://github.com/bits-and-blooms/bloom/releases)
- [Commits](https://github.com/bits-and-blooms/bloom/compare/v3.3.1...v3.5.0)

---
updated-dependencies:
- dependency-name: github.com/bits-and-blooms/bloom/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* feat: add write-delete cache mode

* fix: unnecessary assignment to the blank identifier

* fix: add change into .CHANGELOG file

* build(deps): bump golang.org/x/sync from 0.1.0 to 0.3.0

Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.1.0 to 0.3.0.
- [Commits](https://github.com/golang/sync/compare/v0.1.0...v0.3.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump golang.org/x/crypto

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.0.0-20220315160706-3147a52a75dd to 0.10.0.
- [Commits](https://github.com/golang/crypto/commits/v0.10.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* remove golang--lint-ci

* Beego web.Run() runs the server twice

* fix 5255: Check the rows.Err() if rows.Next() is false

* closes 5254: %COL% should be a common placeholder

* build(deps): bump github.com/prometheus/client_golang

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: use of ioutil package (#5261)

* fix ioutil.NopCloser

* fix ioutil.ReadAll

* fix ioutil.ReadFile

* fix ioutil.WriteFile

* run goimports -w -format-only ./

* update CHANGELOG.md

* feature: add write-double-delete cache mode (#5263)

* cache/redis: support skipEmptyPrefix option (#5264)

* fix: refactor InsertValue method (#5267)

* fix: refactor insertValue method and add the test

* fix: exec goimports and add Licence file header

* fix: modify construct method of dbBase

* fix: add modify record into CHANGELOG

* fix: modify InsertOrUpdate method (#5269)

* fix: modify InsertOrUpdate method, Remove the isMulti variable and its associated code

* fix: Delete unnecessary judgment branches

* fix: add modify record into CHANGELOG

* cache/redis: use redisConfig to receive incoming JSON (previously using a map) (#5268)

* refactor cache/redis: Use redisConfig to receive incoming JSON (previously using a map).

* refactor cache/redis: Use the string type to receive JSON parameters.

---------

Co-authored-by: Tan <tanqianheng@gmail.com>

* fix: refactor Delete method (#5271)

* fix: refactor Delete method and add test

* fix: add modify record into CHANGELOG

* fix: refactor update sql (#5274)

* fix: refactor UpdateSQL method and add test

* fix: add modify record into CHANGELOG

* fix: modify url in the CHANGELOG

* fix: modify pr url in the CHANGELOG

* Fix setPK function for table without primary key (#5276)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: cui fliter <imcusg@gmail.com>
Co-authored-by: Stone-afk <73482944+Stone-afk@users.noreply.github.com>
Co-authored-by: hookokoko <hooko@tju.edu.cn>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: hookokoko <648646891@qq.com>
Co-authored-by: Stone-afk <1711865140@qq.com>
Co-authored-by: chenhaokun <chenhaokun@itiger.com>
Co-authored-by: Xuing <admin@xuing.cn>
Co-authored-by: cui fliter <imcusg@gmail.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
Co-authored-by: uzziah <uzziahlin@gmail.com>
Co-authored-by: Hanjiang Yu <delacroix.yu@gmail.com>
Co-authored-by: Kota <mdryzk64smsh@gmail.com>
Co-authored-by: Uzziah <120019273+uzziahlin@users.noreply.github.com>
Co-authored-by: Handkerchiefs-t <59816423+Handkerchiefs-t@users.noreply.github.com>
Co-authored-by: Tan <tanqianheng@gmail.com>
Co-authored-by: mlgd <mlgd17@gmail.com>
This commit is contained in:
Ming Deng
2023-07-31 23:00:02 +08:00
committed by GitHub
parent 420e11ee63
commit 0bd2df91a1
269 changed files with 3904 additions and 18313 deletions

View File

@@ -1,261 +0,0 @@
// Copyright 2020 beego
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package httplib
import (
"errors"
"net"
"net/http"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
type respCarrier struct {
bytes []byte
}
func (r *respCarrier) SetBytes(bytes []byte) {
r.bytes = bytes
}
func (r *respCarrier) String() string {
return string(r.bytes)
}
func TestOptionWithEnableCookie(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org/",
WithEnableCookie(true))
if err != nil {
t.Fatal(err)
}
v := "smallfish"
resp := &respCarrier{}
err = client.Get(resp, "/cookies/set?k1="+v)
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
err = client.Get(resp, "/cookies")
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
n := strings.Index(resp.String(), v)
if n == -1 {
t.Fatal(v + " not found in cookie")
}
}
func TestOptionWithUserAgent(t *testing.T) {
v := "beego"
client, err := NewClient("test", "http://httpbin.org/",
WithUserAgent(v))
if err != nil {
t.Fatal(err)
}
resp := &respCarrier{}
err = client.Get(resp, "/headers")
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
n := strings.Index(resp.String(), v)
if n == -1 {
t.Fatal(v + " not found in user-agent")
}
}
func TestOptionWithCheckRedirect(t *testing.T) {
client, err := NewClient("test", "https://goolnk.com/33BD2j",
WithCheckRedirect(func(redirectReq *http.Request, redirectVia []*http.Request) error {
return errors.New("Redirect triggered")
}))
if err != nil {
t.Fatal(err)
}
err = client.Get(nil, "")
assert.NotNil(t, err)
}
func TestOptionWithHTTPSetting(t *testing.T) {
v := "beego"
var setting BeegoHTTPSettings
setting.EnableCookie = true
setting.UserAgent = v
setting.Transport = &http.Transport{
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
DualStack: true,
}).DialContext,
MaxIdleConns: 50,
IdleConnTimeout: 90 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
}
setting.ReadWriteTimeout = 5 * time.Second
client, err := NewClient("test", "http://httpbin.org/",
WithHTTPSetting(setting))
if err != nil {
t.Fatal(err)
}
resp := &respCarrier{}
err = client.Get(resp, "/get")
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
n := strings.Index(resp.String(), v)
if n == -1 {
t.Fatal(v + " not found in user-agent")
}
}
func TestOptionWithHeader(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org/")
if err != nil {
t.Fatal(err)
}
client.CommonOpts = append(client.CommonOpts, WithHeader("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36"))
resp := &respCarrier{}
err = client.Get(resp, "/headers")
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
n := strings.Index(resp.String(), "Mozilla/5.0")
if n == -1 {
t.Fatal("Mozilla/5.0 not found in user-agent")
}
}
func TestOptionWithTokenFactory(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org/")
if err != nil {
t.Fatal(err)
}
client.CommonOpts = append(client.CommonOpts,
WithTokenFactory(func() string {
return "testauth"
}))
resp := &respCarrier{}
err = client.Get(resp, "/headers")
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
n := strings.Index(resp.String(), "testauth")
if n == -1 {
t.Fatal("Auth is not set in request")
}
}
func TestOptionWithBasicAuth(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org/")
if err != nil {
t.Fatal(err)
}
resp := &respCarrier{}
err = client.Get(resp, "/basic-auth/user/passwd",
WithBasicAuth(func() (string, string) {
return "user", "passwd"
}))
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
n := strings.Index(resp.String(), "authenticated")
if n == -1 {
t.Fatal("authenticated not found in response")
}
}
func TestOptionWithContentType(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org/")
if err != nil {
t.Fatal(err)
}
v := "application/json"
resp := &respCarrier{}
err = client.Get(resp, "/headers", WithContentType(v))
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
n := strings.Index(resp.String(), v)
if n == -1 {
t.Fatal(v + " not found in header")
}
}
func TestOptionWithParam(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org/")
if err != nil {
t.Fatal(err)
}
v := "smallfish"
resp := &respCarrier{}
err = client.Get(resp, "/get", WithParam("username", v))
if err != nil {
t.Fatal(err)
}
t.Log(resp.String())
n := strings.Index(resp.String(), v)
if n == -1 {
t.Fatal(v + " not found in header")
}
}
func TestOptionWithRetry(t *testing.T) {
client, err := NewClient("test", "https://goolnk.com/33BD2j",
WithCheckRedirect(func(redirectReq *http.Request, redirectVia []*http.Request) error {
return errors.New("Redirect triggered")
}))
if err != nil {
t.Fatal(err)
}
retryAmount := 1
retryDelay := 800 * time.Millisecond
startTime := time.Now().UnixNano() / int64(time.Millisecond)
_ = client.Get(nil, "", WithRetry(retryAmount, retryDelay))
endTime := time.Now().UnixNano() / int64(time.Millisecond)
elapsedTime := endTime - startTime
delayedTime := int64(retryAmount) * retryDelay.Milliseconds()
if elapsedTime < delayedTime {
t.Errorf("Not enough retries. Took %dms. Delay was meant to take %dms", elapsedTime, delayedTime)
}
}

View File

@@ -18,8 +18,8 @@ import (
"context"
"net/http"
logKit "github.com/go-kit/kit/log"
opentracingKit "github.com/go-kit/kit/tracing/opentracing"
logKit "github.com/go-kit/log"
"github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/log"

View File

@@ -17,7 +17,7 @@ package httplib
import (
"bytes"
"encoding/json"
"io/ioutil"
"io"
"net/http"
)
@@ -34,6 +34,6 @@ func NewHttpResponseWithJsonBody(data interface{}) *http.Response {
}
return &http.Response{
ContentLength: int64(len(body)),
Body: ioutil.NopCloser(bytes.NewReader(body)),
Body: io.NopCloser(bytes.NewReader(body)),
}
}

View File

@@ -17,7 +17,6 @@ package httplib
import (
"bytes"
"io"
"io/ioutil"
"net/http"
)
@@ -105,7 +104,7 @@ func (c *Client) handleCarrier(value interface{}, req *BeegoHTTPRequest) error {
if err != nil {
return err
}
req.resp.Body = ioutil.NopCloser(bytes.NewReader(b))
req.resp.Body = io.NopCloser(bytes.NewReader(b))
carrier.SetHTTPResponse(req.resp)
}
if carrier, ok := value.(HTTPBodyCarrier); ok {
@@ -113,7 +112,7 @@ func (c *Client) handleCarrier(value interface{}, req *BeegoHTTPRequest) error {
if err != nil {
return err
}
reader := ioutil.NopCloser(bytes.NewReader(b))
reader := io.NopCloser(bytes.NewReader(b))
carrier.SetReader(reader)
}
if carrier, ok := value.(HTTPBytesCarrier); ok {

View File

@@ -15,12 +15,17 @@
package httplib
import (
"encoding/json"
"encoding/xml"
"io"
"io/ioutil"
"net"
"net/http"
"testing"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
"gopkg.in/yaml.v3"
"github.com/stretchr/testify/assert"
)
@@ -32,13 +37,13 @@ func TestNewClient(t *testing.T) {
}
type slideShowResponse struct {
Resp *http.Response
bytes []byte
StatusCode int
Body io.ReadCloser
Header map[string][]string
Resp *http.Response `json:"resp,omitempty"`
bytes []byte `json:"bytes,omitempty"`
StatusCode int `json:"status_code,omitempty"`
Body io.ReadCloser `json:"body,omitempty"`
Header map[string][]string `json:"header,omitempty"`
Slideshow slideshow `json:"slideshow" yaml:"slideshow"`
Slideshow slideshow `json:"slideshow,omitempty" yaml:"slideshow" xml:"slideshow"`
}
func (r *slideShowResponse) SetHTTPResponse(resp *http.Response) {
@@ -66,7 +71,7 @@ func (r *slideShowResponse) String() string {
}
type slideshow struct {
XMLName xml.Name `xml:"slideshow"`
//XMLName xml.Name `xml:"slideshow"`
Title string `json:"title" yaml:"title" xml:"title,attr"`
Author string `json:"author" yaml:"author" xml:"author,attr"`
@@ -80,63 +85,122 @@ type slide struct {
Title string `json:"title" yaml:"title" xml:"title"`
}
func TestClientHandleCarrier(t *testing.T) {
v := "beego"
client, err := NewClient("test", "http://httpbin.org/",
WithUserAgent(v))
if err != nil {
t.Fatal(err)
}
s := &slideShowResponse{}
err = client.Get(s, "/json")
if err != nil {
t.Fatal(err)
}
defer s.Body.Close()
assert.NotNil(t, s.Resp)
assert.NotNil(t, s.Body)
assert.Equal(t, "429", s.Header["Content-Length"][0])
assert.Equal(t, 200, s.StatusCode)
b, err := ioutil.ReadAll(s.Body)
if err != nil {
t.Fatal(err)
}
assert.Equal(t, 429, len(b))
assert.Equal(t, s.String(), string(b))
type ClientTestSuite struct {
suite.Suite
l net.Listener
}
func TestClientGet(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org/")
func (c *ClientTestSuite) SetupSuite() {
listener, err := net.Listen("tcp", ":8080")
require.NoError(c.T(), err)
c.l = listener
handler := http.NewServeMux()
handler.HandleFunc("/json", func(writer http.ResponseWriter, request *http.Request) {
data, _ := json.Marshal(slideshow{})
_, _ = writer.Write(data)
})
ssr := slideShowResponse{
Slideshow: slideshow{
Title: "Sample Slide Show",
Slides: []slide{
{
Title: "Content",
},
{
Title: "Overview",
},
},
},
}
handler.HandleFunc("/req2resp", func(writer http.ResponseWriter, request *http.Request) {
data, _ := io.ReadAll(request.Body)
_, _ = writer.Write(data)
})
handler.HandleFunc("/get", func(writer http.ResponseWriter, request *http.Request) {
data, _ := json.Marshal(ssr)
_, _ = writer.Write(data)
})
handler.HandleFunc("/get/xml", func(writer http.ResponseWriter, request *http.Request) {
data, err := xml.Marshal(ssr.Slideshow)
require.NoError(c.T(), err)
_, _ = writer.Write(data)
})
handler.HandleFunc("/get/yaml", func(writer http.ResponseWriter, request *http.Request) {
data, _ := yaml.Marshal(ssr)
_, _ = writer.Write(data)
})
go func() {
_ = http.Serve(listener, handler)
}()
}
func (c *ClientTestSuite) TearDownSuite() {
_ = c.l.Close()
}
func TestClient(t *testing.T) {
suite.Run(t, &ClientTestSuite{})
}
func (c *ClientTestSuite) TestClientHandleCarrier() {
t := c.T()
v := "beego"
client, err := NewClient("test", "http://localhost:8080/",
WithUserAgent(v))
require.NoError(t, err)
resp := &slideShowResponse{}
err = client.Get(resp, "/json")
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
assert.NotNil(t, resp.Resp)
assert.NotNil(t, resp.Body)
assert.Equal(t, "48", resp.Header["Content-Length"][0])
assert.Equal(t, 200, resp.StatusCode)
b, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatal(err)
}
assert.Equal(t, 48, len(b))
assert.Equal(t, resp.String(), string(b))
}
func (c *ClientTestSuite) TestClientGet() {
t := c.T()
client, err := NewClient("test", "http://localhost:8080/")
if err != nil {
t.Fatal(err)
}
// json
var s *slideShowResponse
err = client.Get(&s, "/json")
if err != nil {
t.Fatal(err)
}
var s slideShowResponse
err = client.Get(&s, "/get")
require.NoError(t, err)
assert.Equal(t, "Sample Slide Show", s.Slideshow.Title)
assert.Equal(t, 2, len(s.Slideshow.Slides))
assert.Equal(t, "Overview", s.Slideshow.Slides[1].Title)
// xml
var ssp *slideshow
err = client.Get(&ssp, "/base64/PD94bWwgPz48c2xpZGVzaG93CnRpdGxlPSJTYW1wbGUgU2xpZGUgU2hvdyIKZGF0ZT0iRGF0ZSBvZiBwdWJsaWNhdGlvbiIKYXV0aG9yPSJZb3VycyBUcnVseSI+PHNsaWRlIHR5cGU9ImFsbCI+PHRpdGxlPldha2UgdXAgdG8gV29uZGVyV2lkZ2V0cyE8L3RpdGxlPjwvc2xpZGU+PHNsaWRlIHR5cGU9ImFsbCI+PHRpdGxlPk92ZXJ2aWV3PC90aXRsZT48aXRlbT5XaHkgPGVtPldvbmRlcldpZGdldHM8L2VtPiBhcmUgZ3JlYXQ8L2l0ZW0+PGl0ZW0vPjxpdGVtPldobyA8ZW0+YnV5czwvZW0+IFdvbmRlcldpZGdldHM8L2l0ZW0+PC9zbGlkZT48L3NsaWRlc2hvdz4=")
if err != nil {
t.Fatal(err)
}
assert.Equal(t, "Sample Slide Show", ssp.Title)
assert.Equal(t, 2, len(ssp.Slides))
assert.Equal(t, "Overview", ssp.Slides[1].Title)
var ss slideshow
err = client.Get(&ss, "/get/xml")
require.NoError(t, err)
assert.Equal(t, "Sample Slide Show", ss.Title)
assert.Equal(t, 2, len(ss.Slides))
assert.Equal(t, "Overview", ss.Slides[1].Title)
// yaml
s = nil
err = client.Get(&s, "/base64/c2xpZGVzaG93OgogIGF1dGhvcjogWW91cnMgVHJ1bHkKICBkYXRlOiBkYXRlIG9mIHB1YmxpY2F0aW9uCiAgc2xpZGVzOgogIC0gdGl0bGU6IFdha2UgdXAgdG8gV29uZGVyV2lkZ2V0cyEKICAgIHR5cGU6IGFsbAogIC0gaXRlbXM6CiAgICAtIFdoeSA8ZW0+V29uZGVyV2lkZ2V0czwvZW0+IGFyZSBncmVhdAogICAgLSBXaG8gPGVtPmJ1eXM8L2VtPiBXb25kZXJXaWRnZXRzCiAgICB0aXRsZTogT3ZlcnZpZXcKICAgIHR5cGU6IGFsbAogIHRpdGxlOiBTYW1wbGUgU2xpZGUgU2hvdw==")
s = slideShowResponse{}
err = client.Get(&s, "/get/yaml")
if err != nil {
t.Fatal(err)
}
@@ -145,72 +209,82 @@ func TestClientGet(t *testing.T) {
assert.Equal(t, "Overview", s.Slideshow.Slides[1].Title)
}
func TestClientPost(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org")
if err != nil {
t.Fatal(err)
func (c *ClientTestSuite) TestClientPost() {
t := c.T()
client, err := NewClient("test", "http://localhost:8080")
require.NoError(t, err)
input := slideShowResponse{
Slideshow: slideshow{
Title: "Sample Slide Show",
Slides: []slide{
{
Title: "Content",
},
{
Title: "Overview",
},
},
},
}
resp := &slideShowResponse{}
err = client.Get(resp, "/json")
if err != nil {
t.Fatal(err)
}
jsonStr := resp.String()
err = client.Post(resp, "/post", jsonStr)
if err != nil {
t.Fatal(err)
}
assert.NotNil(t, resp)
jsonStr, err := json.Marshal(input)
require.NoError(t, err)
resp := slideShowResponse{}
err = client.Post(&resp, "/req2resp", jsonStr)
require.NoError(t, err)
assert.Equal(t, input.Slideshow, resp.Slideshow)
assert.Equal(t, http.MethodPost, resp.Resp.Request.Method)
}
func TestClientPut(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org")
if err != nil {
t.Fatal(err)
func (c *ClientTestSuite) TestClientPut() {
t := c.T()
client, err := NewClient("test", "http://localhost:8080")
require.NoError(t, err)
input := slideShowResponse{
Slideshow: slideshow{
Title: "Sample Slide Show",
Slides: []slide{
{
Title: "Content",
},
{
Title: "Overview",
},
},
},
}
resp := &slideShowResponse{}
err = client.Get(resp, "/json")
if err != nil {
t.Fatal(err)
}
jsonStr := resp.String()
err = client.Put(resp, "/put", jsonStr)
if err != nil {
t.Fatal(err)
}
assert.NotNil(t, resp)
jsonStr, err := json.Marshal(input)
require.NoError(t, err)
resp := slideShowResponse{}
err = client.Put(&resp, "/req2resp", jsonStr)
require.NoError(t, err)
assert.Equal(t, input.Slideshow, resp.Slideshow)
assert.Equal(t, http.MethodPut, resp.Resp.Request.Method)
}
func TestClientDelete(t *testing.T) {
client, err := NewClient("test", "http://httpbin.org")
if err != nil {
t.Fatal(err)
}
func (c *ClientTestSuite) TestClientDelete() {
t := c.T()
client, err := NewClient("test", "http://localhost:8080")
require.NoError(t, err)
resp := &slideShowResponse{}
err = client.Delete(resp, "/delete")
if err != nil {
t.Fatal(err)
}
err = client.Delete(resp, "/req2resp")
require.NoError(t, err)
defer resp.Resp.Body.Close()
assert.NotNil(t, resp)
assert.Equal(t, http.MethodDelete, resp.Resp.Request.Method)
}
func TestClientHead(t *testing.T) {
client, err := NewClient("test", "http://beego.gocn.vip")
if err != nil {
t.Fatal(err)
}
func (c *ClientTestSuite) TestClientHead() {
t := c.T()
client, err := NewClient("test", "http://localhost:8080")
require.NoError(t, err)
resp := &slideShowResponse{}
err = client.Head(resp, "")
err = client.Head(resp, "/req2resp")
if err != nil {
t.Fatal(err)
}

View File

@@ -27,7 +27,6 @@
// t.Fatal(err)
// }
// fmt.Println(str)
//
package httplib
import (
@@ -38,7 +37,6 @@ import (
"encoding/json"
"encoding/xml"
"io"
"io/ioutil"
"mime/multipart"
"net"
"net/http"
@@ -225,9 +223,9 @@ func (b *BeegoHTTPRequest) SetTransport(transport http.RoundTripper) *BeegoHTTPR
// example:
//
// func(req *http.Request) (*url.URL, error) {
// u, _ := url.ParseRequestURI("http://127.0.0.1:8118")
// return u, nil
// }
// u, _ := url.ParseRequestURI("http://127.0.0.1:8118")
// return u, nil
// }
func (b *BeegoHTTPRequest) SetProxy(proxy func(*http.Request) (*url.URL, error)) *BeegoHTTPRequest {
b.setting.Proxy = proxy
return b
@@ -284,16 +282,16 @@ func (b *BeegoHTTPRequest) Body(data interface{}) *BeegoHTTPRequest {
switch t := data.(type) {
case string:
bf := bytes.NewBufferString(t)
b.req.Body = ioutil.NopCloser(bf)
b.req.Body = io.NopCloser(bf)
b.req.GetBody = func() (io.ReadCloser, error) {
return ioutil.NopCloser(bf), nil
return io.NopCloser(bf), nil
}
b.req.ContentLength = int64(len(t))
case []byte:
bf := bytes.NewBuffer(t)
b.req.Body = ioutil.NopCloser(bf)
b.req.Body = io.NopCloser(bf)
b.req.GetBody = func() (io.ReadCloser, error) {
return ioutil.NopCloser(bf), nil
return io.NopCloser(bf), nil
}
b.req.ContentLength = int64(len(t))
default:
@@ -309,9 +307,9 @@ func (b *BeegoHTTPRequest) XMLBody(obj interface{}) (*BeegoHTTPRequest, error) {
if err != nil {
return b, berror.Wrap(err, InvalidXMLBody, "obj could not be converted to XML data")
}
b.req.Body = ioutil.NopCloser(bytes.NewReader(byts))
b.req.Body = io.NopCloser(bytes.NewReader(byts))
b.req.GetBody = func() (io.ReadCloser, error) {
return ioutil.NopCloser(bytes.NewReader(byts)), nil
return io.NopCloser(bytes.NewReader(byts)), nil
}
b.req.ContentLength = int64(len(byts))
b.req.Header.Set(contentTypeKey, "application/xml")
@@ -326,7 +324,7 @@ func (b *BeegoHTTPRequest) YAMLBody(obj interface{}) (*BeegoHTTPRequest, error)
if err != nil {
return b, berror.Wrap(err, InvalidYAMLBody, "obj could not be converted to YAML data")
}
b.req.Body = ioutil.NopCloser(bytes.NewReader(byts))
b.req.Body = io.NopCloser(bytes.NewReader(byts))
b.req.ContentLength = int64(len(byts))
b.req.Header.Set(contentTypeKey, "application/x+yaml")
}
@@ -340,7 +338,7 @@ func (b *BeegoHTTPRequest) JSONBody(obj interface{}) (*BeegoHTTPRequest, error)
if err != nil {
return b, berror.Wrap(err, InvalidJSONBody, "obj could not be converted to JSON body")
}
b.req.Body = ioutil.NopCloser(bytes.NewReader(byts))
b.req.Body = io.NopCloser(bytes.NewReader(byts))
b.req.ContentLength = int64(len(byts))
b.req.Header.Set(contentTypeKey, "application/json")
}
@@ -401,7 +399,7 @@ func (b *BeegoHTTPRequest) handleFiles() {
_ = pw.Close()
}()
b.Header(contentTypeKey, bodyWriter.FormDataContentType())
b.req.Body = ioutil.NopCloser(pr)
b.req.Body = io.NopCloser(pr)
b.Header("Transfer-Encoding", "chunked")
}
@@ -592,10 +590,10 @@ func (b *BeegoHTTPRequest) Bytes() ([]byte, error) {
if err != nil {
return nil, berror.Wrap(err, ReadGzipBodyFailed, "building gzip reader failed")
}
b.body, err = ioutil.ReadAll(reader)
b.body, err = io.ReadAll(reader)
return b.body, berror.Wrap(err, ReadGzipBodyFailed, "reading gzip data failed")
}
b.body, err = ioutil.ReadAll(resp.Body)
b.body, err = io.ReadAll(resp.Body)
return b.body, err
}

View File

@@ -17,9 +17,10 @@ package httplib
import (
"bytes"
"context"
json "encoding/json"
"errors"
"fmt"
"io/ioutil"
"io"
"net"
"net/http"
"os"
@@ -27,19 +28,104 @@ import (
"testing"
"time"
"github.com/stretchr/testify/suite"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestResponse(t *testing.T) {
req := Get("http://httpbin.org/get")
resp, err := req.Response()
require.NoError(t, err)
t.Log(resp)
type HttplibTestSuite struct {
suite.Suite
l net.Listener
}
func TestDoRequest(t *testing.T) {
req := Get("https://goolnk.com/33BD2j")
func (h *HttplibTestSuite) SetupSuite() {
listener, err := net.Listen("tcp", ":8080")
require.NoError(h.T(), err)
h.l = listener
handler := http.NewServeMux()
handler.HandleFunc("/get", func(writer http.ResponseWriter, request *http.Request) {
agent := request.Header.Get("User-Agent")
_, _ = writer.Write([]byte("hello, " + agent))
})
handler.HandleFunc("/put", func(writer http.ResponseWriter, request *http.Request) {
_, _ = writer.Write([]byte("hello, put"))
})
handler.HandleFunc("/post", func(writer http.ResponseWriter, request *http.Request) {
body, _ := io.ReadAll(request.Body)
_, _ = writer.Write(body)
})
handler.HandleFunc("/delete", func(writer http.ResponseWriter, request *http.Request) {
_, _ = writer.Write([]byte("hello, delete"))
})
handler.HandleFunc("/cookies/set", func(writer http.ResponseWriter, request *http.Request) {
k1 := request.URL.Query().Get("k1")
http.SetCookie(writer, &http.Cookie{
Name: "k1",
Value: k1,
})
_, _ = writer.Write([]byte("hello, set cookie"))
})
handler.HandleFunc("/cookies", func(writer http.ResponseWriter, request *http.Request) {
body := request.Cookies()[0].String()
_, _ = writer.Write([]byte(body))
})
handler.HandleFunc("/basic-auth/user/passwd", func(writer http.ResponseWriter, request *http.Request) {
_, _, ok := request.BasicAuth()
if ok {
_, _ = writer.Write([]byte("authenticated"))
} else {
_, _ = writer.Write([]byte("no auth"))
}
})
handler.HandleFunc("/headers", func(writer http.ResponseWriter, request *http.Request) {
agent := request.Header.Get("User-Agent")
_, _ = writer.Write([]byte(agent))
})
handler.HandleFunc("/ip", func(writer http.ResponseWriter, request *http.Request) {
data := map[string]string{"origin": "127.0.0.1"}
jsonBytes, _ := json.Marshal(data)
_, _ = writer.Write(jsonBytes)
})
handler.HandleFunc("/redirect", func(writer http.ResponseWriter, request *http.Request) {
http.Redirect(writer, request, "redirect_dst", http.StatusTemporaryRedirect)
})
handler.HandleFunc("redirect_dst", func(writer http.ResponseWriter, request *http.Request) {
_, _ = writer.Write([]byte("hello"))
})
go func() {
_ = http.Serve(listener, handler)
}()
}
func (h *HttplibTestSuite) TearDownSuite() {
_ = h.l.Close()
}
func TestHttplib(t *testing.T) {
suite.Run(t, &HttplibTestSuite{})
}
func (h *HttplibTestSuite) TestResponse() {
req := Get("http://localhost:8080/get")
_, err := req.Response()
require.NoError(h.T(), err)
}
func (h *HttplibTestSuite) TestDoRequest() {
t := h.T()
req := Get("http://localhost:8080/redirect")
retryAmount := 1
req.Retries(1)
req.RetryDelay(1400 * time.Millisecond)
@@ -63,107 +149,79 @@ func TestDoRequest(t *testing.T) {
}
}
func TestGet(t *testing.T) {
req := Get("http://httpbin.org/get")
func (h *HttplibTestSuite) TestGet() {
t := h.T()
req := Get("http://localhost:8080/get")
b, err := req.Bytes()
require.NoError(t, err)
t.Log(b)
s, err := req.String()
require.NoError(t, err)
t.Log(s)
require.Equal(t, string(b), s)
}
func TestSimplePost(t *testing.T) {
func (h *HttplibTestSuite) TestSimplePost() {
t := h.T()
v := "smallfish"
req := Post("http://httpbin.org/post")
req := Post("http://localhost:8080/post")
req.Param("username", v)
str, err := req.String()
require.NoError(t, err)
t.Log(str)
n := strings.Index(str, v)
require.NotEqual(t, -1, n)
}
func (h *HttplibTestSuite) TestSimplePut() {
t := h.T()
_, err := Put("http://localhost:8080/put").String()
require.NoError(t, err)
}
func (h *HttplibTestSuite) TestSimpleDelete() {
t := h.T()
_, err := Delete("http://localhost:8080/delete").String()
require.NoError(t, err)
}
func (h *HttplibTestSuite) TestSimpleDeleteParam() {
t := h.T()
_, err := Delete("http://localhost:8080/delete").Param("key", "val").String()
require.NoError(t, err)
}
func (h *HttplibTestSuite) TestWithCookie() {
t := h.T()
v := "smallfish"
_, err := Get("http://localhost:8080/cookies/set?k1=" + v).SetEnableCookie(true).String()
require.NoError(t, err)
str, err := Get("http://localhost:8080/cookies").SetEnableCookie(true).String()
require.NoError(t, err)
n := strings.Index(str, v)
require.NotEqual(t, -1, n)
}
// func TestPostFile(t *testing.T) {
// v := "smallfish"
// req := Post("http://httpbin.org/post")
// req.Debug(true)
// req.Param("username", v)
// req.PostFile("uploadfile", "httplib_test.go")
// str, err := req.String()
// if err != nil {
// t.Fatal(err)
// }
// t.Log(str)
// n := strings.Index(str, v)
// if n == -1 {
// t.Fatal(v + " not found in post")
// }
// }
func TestSimplePut(t *testing.T) {
str, err := Put("http://httpbin.org/put").String()
func (h *HttplibTestSuite) TestWithBasicAuth() {
t := h.T()
str, err := Get("http://localhost:8080/basic-auth/user/passwd").SetBasicAuth("user", "passwd").String()
require.NoError(t, err)
t.Log(str)
}
func TestSimpleDelete(t *testing.T) {
str, err := Delete("http://httpbin.org/delete").String()
require.NoError(t, err)
t.Log(str)
}
func TestSimpleDeleteParam(t *testing.T) {
str, err := Delete("http://httpbin.org/delete").Param("key", "val").String()
require.NoError(t, err)
t.Log(str)
}
func TestWithCookie(t *testing.T) {
v := "smallfish"
str, err := Get("http://httpbin.org/cookies/set?k1=" + v).SetEnableCookie(true).String()
require.NoError(t, err)
t.Log(str)
str, err = Get("http://httpbin.org/cookies").SetEnableCookie(true).String()
require.NoError(t, err)
t.Log(str)
n := strings.Index(str, v)
if n == -1 {
t.Fatal(v + " not found in cookie")
}
}
func TestWithBasicAuth(t *testing.T) {
str, err := Get("http://httpbin.org/basic-auth/user/passwd").SetBasicAuth("user", "passwd").String()
require.NoError(t, err)
t.Log(str)
n := strings.Index(str, "authenticated")
if n == -1 {
t.Fatal("authenticated not found in response")
}
require.NotEqual(t, -1, n)
}
func TestWithUserAgent(t *testing.T) {
func (h *HttplibTestSuite) TestWithUserAgent() {
t := h.T()
v := "beego"
str, err := Get("http://httpbin.org/headers").SetUserAgent(v).String()
str, err := Get("http://localhost:8080/headers").SetUserAgent(v).String()
require.NoError(t, err)
t.Log(str)
n := strings.Index(str, v)
if n == -1 {
t.Fatal(v + " not found in user-agent")
}
require.NotEqual(t, -1, n)
}
func TestWithSetting(t *testing.T) {
func (h *HttplibTestSuite) TestWithSetting() {
t := h.T()
v := "beego"
var setting BeegoHTTPSettings
setting.EnableCookie = true
@@ -181,75 +239,68 @@ func TestWithSetting(t *testing.T) {
setting.ReadWriteTimeout = 5 * time.Second
SetDefaultSetting(setting)
str, err := Get("http://httpbin.org/get").String()
str, err := Get("http://localhost:8080/get").String()
require.NoError(t, err)
t.Log(str)
n := strings.Index(str, v)
if n == -1 {
t.Fatal(v + " not found in user-agent")
}
require.NotEqual(t, -1, n)
}
func TestToJson(t *testing.T) {
req := Get("http://httpbin.org/ip")
func (h *HttplibTestSuite) TestToJson() {
t := h.T()
req := Get("http://localhost:8080/ip")
resp, err := req.Response()
require.NoError(t, err)
t.Log(resp)
// httpbin will return http remote addr
type IP struct {
Origin string `json:"origin"`
}
var ip IP
err = req.ToJSON(&ip)
require.NoError(t, err)
t.Log(ip.Origin)
require.Equal(t, "127.0.0.1", ip.Origin)
ips := strings.Split(ip.Origin, ",")
if len(ips) == 0 {
t.Fatal("response is not valid ip")
}
for i := range ips {
if net.ParseIP(strings.TrimSpace(ips[i])).To4() == nil {
t.Fatal("response is not valid ip")
}
}
require.NotEmpty(t, ips)
}
func TestToFile(t *testing.T) {
func (h *HttplibTestSuite) TestToFile() {
t := h.T()
f := "beego_testfile"
req := Get("http://httpbin.org/ip")
req := Get("http://localhost:8080/ip")
err := req.ToFile(f)
require.NoError(t, err)
defer os.Remove(f)
b, err := ioutil.ReadFile(f)
if n := bytes.Index(b, []byte("origin")); n == -1 {
t.Fatal(err)
}
b, err := os.ReadFile(f)
n := bytes.Index(b, []byte("origin"))
require.NotEqual(t, -1, n)
}
func TestToFileDir(t *testing.T) {
func (h *HttplibTestSuite) TestToFileDir() {
t := h.T()
f := "./files/beego_testfile"
req := Get("http://httpbin.org/ip")
req := Get("http://localhost:8080/ip")
err := req.ToFile(f)
require.NoError(t, err)
defer os.RemoveAll("./files")
b, err := ioutil.ReadFile(f)
if n := bytes.Index(b, []byte("origin")); n == -1 {
t.Fatal(err)
}
b, err := os.ReadFile(f)
require.NoError(t, err)
n := bytes.Index(b, []byte("origin"))
require.NotEqual(t, -1, n)
}
func TestHeader(t *testing.T) {
req := Get("http://httpbin.org/headers")
func (h *HttplibTestSuite) TestHeader() {
t := h.T()
req := Get("http://localhost:8080/headers")
req.Header("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36")
str, err := req.String()
_, err := req.String()
require.NoError(t, err)
t.Log(str)
}
// TestAddFilter make sure that AddFilters only work for the specific request
func TestAddFilter(t *testing.T) {
func (h *HttplibTestSuite) TestAddFilter() {
t := h.T()
req := Get("http://beego.vip")
req.AddFilters(func(next Filter) Filter {
return func(ctx context.Context, req *BeegoHTTPRequest) (*http.Response, error) {
@@ -261,7 +312,8 @@ func TestAddFilter(t *testing.T) {
assert.Equal(t, 1, len(req.setting.FilterChains)-len(r.setting.FilterChains))
}
func TestFilterChainOrder(t *testing.T) {
func (h *HttplibTestSuite) TestFilterChainOrder() {
t := h.T()
req := Get("http://beego.vip")
req.AddFilters(func(next Filter) Filter {
return func(ctx context.Context, req *BeegoHTTPRequest) (*http.Response, error) {
@@ -282,24 +334,34 @@ func TestFilterChainOrder(t *testing.T) {
assert.Equal(t, "first", string(data))
}
func TestHead(t *testing.T) {
func (h *HttplibTestSuite) TestHead() {
t := h.T()
req := Head("http://beego.vip")
assert.NotNil(t, req)
assert.Equal(t, "HEAD", req.req.Method)
}
func TestDelete(t *testing.T) {
func (h *HttplibTestSuite) TestDelete() {
t := h.T()
req := Delete("http://beego.vip")
assert.NotNil(t, req)
assert.Equal(t, "DELETE", req.req.Method)
}
func TestPost(t *testing.T) {
func (h *HttplibTestSuite) TestPost() {
t := h.T()
req := Post("http://beego.vip")
assert.NotNil(t, req)
assert.Equal(t, "POST", req.req.Method)
}
func (h *HttplibTestSuite) TestPut() {
t := h.T()
req := Put("http://beego.vip")
assert.NotNil(t, req)
assert.Equal(t, "PUT", req.req.Method)
}
func TestNewBeegoRequest(t *testing.T) {
req := NewBeegoRequest("http://beego.vip", "GET")
assert.NotNil(t, req)
@@ -341,12 +403,6 @@ func TestBeegoHTTPRequestSetProtocolVersion(t *testing.T) {
assert.Equal(t, 1, req.req.ProtoMinor)
}
func TestPut(t *testing.T) {
req := Put("http://beego.vip")
assert.NotNil(t, req)
assert.Equal(t, "PUT", req.req.Method)
}
func TestBeegoHTTPRequestHeader(t *testing.T) {
req := Post("http://beego.vip")
key, value := "test-header", "test-header-value"