Ming Deng 0bd2df91a1
Resolve conflicts among master branch and develop branch (#5286)
* feature extend readthrough for cache module (#5116)

* feature 增加readthrough

* feature: add write though for cache mode (#5117)

* feature: add writethough for cache mode

* feature add singleflight cache (#5119)

* build(deps): bump go.opentelemetry.io/otel/trace from 1.8.0 to 1.11.2

Bumps [go.opentelemetry.io/otel/trace](https://github.com/open-telemetry/opentelemetry-go) from 1.8.0 to 1.11.2.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.8.0...v1.11.2)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/trace
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix 5129: must set formatter after init the logger

* remove beego.vip

* build(deps): bump actions/stale from 5 to 7

Bumps [actions/stale](https://github.com/actions/stale) from 5 to 7.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix 5079: only log msg when the channel is not closed (#5132)

* optimize test

* upgrade otel dependencies to v1.11.2

* format code

* Bloom filter cache (#5126)

* feature: add bloom filter cache

* feature upload remove all temp file

* bugfix Controller SaveToFile remove all temp file

* rft: motify BeeLogger signalChan (#5139)

* add non-block write log in asynchronous mode (#5150)

* add non-block write log in asynchronous mode

---------

Co-authored-by: chenhaokun <chenhaokun@itiger.com>

* fix the docsite URL (#5173)

* Unified gopkg.in/yaml version to v2 (#5169)

* Unified gopkg.in/yaml version to v2 and go mod tidy

* update CHANGELOG

* bugfix: protect field access with lock to avoid possible data race (#5211)

* fix some comments (#5194)

Signed-off-by: cui fliter <imcusg@gmail.com>

* build(deps): bump github.com/prometheus/client_golang (#5213)

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.14.0 to 1.15.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.14.0...v1.15.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump go.etcd.io/etcd/client/v3 from 3.5.4 to 3.5.9 (#5209)

Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.5.4 to 3.5.9.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.4...v3.5.9)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* cache: fix typo and optimize the naming

* Release 2.1.0 change log

* bugfix: beegoAppConfig String and Strings function has bug

* httplib: fix unstable test, do not use httplib.org

* chore: pkg imported more than once

* chore: fmt modify

* chore: Use github.com/go-kit/log

* chore: unnecessary use of fmt.Sprintf

* fix: golangci-lint error

* orm: refactor ORM introducing internal/models pkg

* remove adapter package

* build(deps): bump github.com/bits-and-blooms/bloom/v3

Bumps [github.com/bits-and-blooms/bloom/v3](https://github.com/bits-and-blooms/bloom) from 3.3.1 to 3.5.0.
- [Release notes](https://github.com/bits-and-blooms/bloom/releases)
- [Commits](https://github.com/bits-and-blooms/bloom/compare/v3.3.1...v3.5.0)

---
updated-dependencies:
- dependency-name: github.com/bits-and-blooms/bloom/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* feat: add write-delete cache mode

* fix: unnecessary assignment to the blank identifier

* fix: add change into .CHANGELOG file

* build(deps): bump golang.org/x/sync from 0.1.0 to 0.3.0

Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.1.0 to 0.3.0.
- [Commits](https://github.com/golang/sync/compare/v0.1.0...v0.3.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump golang.org/x/crypto

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.0.0-20220315160706-3147a52a75dd to 0.10.0.
- [Commits](https://github.com/golang/crypto/commits/v0.10.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* remove golang--lint-ci

* Beego web.Run() runs the server twice

* fix 5255: Check the rows.Err() if rows.Next() is false

* closes 5254: %COL% should be a common placeholder

* build(deps): bump github.com/prometheus/client_golang

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: use of ioutil package (#5261)

* fix ioutil.NopCloser

* fix ioutil.ReadAll

* fix ioutil.ReadFile

* fix ioutil.WriteFile

* run goimports -w -format-only ./

* update CHANGELOG.md

* feature: add write-double-delete cache mode (#5263)

* cache/redis: support skipEmptyPrefix option (#5264)

* fix: refactor InsertValue method (#5267)

* fix: refactor insertValue method and add the test

* fix: exec goimports and add Licence file header

* fix: modify construct method of dbBase

* fix: add modify record into CHANGELOG

* fix: modify InsertOrUpdate method (#5269)

* fix: modify InsertOrUpdate method, Remove the isMulti variable and its associated code

* fix: Delete unnecessary judgment branches

* fix: add modify record into CHANGELOG

* cache/redis: use redisConfig to receive incoming JSON (previously using a map) (#5268)

* refactor cache/redis: Use redisConfig to receive incoming JSON (previously using a map).

* refactor cache/redis: Use the string type to receive JSON parameters.

---------

Co-authored-by: Tan <tanqianheng@gmail.com>

* fix: refactor Delete method (#5271)

* fix: refactor Delete method and add test

* fix: add modify record into CHANGELOG

* fix: refactor update sql (#5274)

* fix: refactor UpdateSQL method and add test

* fix: add modify record into CHANGELOG

* fix: modify url in the CHANGELOG

* fix: modify pr url in the CHANGELOG

* Fix setPK function for table without primary key (#5276)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: cui fliter <imcusg@gmail.com>
Co-authored-by: Stone-afk <73482944+Stone-afk@users.noreply.github.com>
Co-authored-by: hookokoko <hooko@tju.edu.cn>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: hookokoko <648646891@qq.com>
Co-authored-by: Stone-afk <1711865140@qq.com>
Co-authored-by: chenhaokun <chenhaokun@itiger.com>
Co-authored-by: Xuing <admin@xuing.cn>
Co-authored-by: cui fliter <imcusg@gmail.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
Co-authored-by: uzziah <uzziahlin@gmail.com>
Co-authored-by: Hanjiang Yu <delacroix.yu@gmail.com>
Co-authored-by: Kota <mdryzk64smsh@gmail.com>
Co-authored-by: Uzziah <120019273+uzziahlin@users.noreply.github.com>
Co-authored-by: Handkerchiefs-t <59816423+Handkerchiefs-t@users.noreply.github.com>
Co-authored-by: Tan <tanqianheng@gmail.com>
Co-authored-by: mlgd <mlgd17@gmail.com>
2023-07-31 23:00:02 +08:00

301 lines
6.6 KiB
Go

// Copyright 2014 beego Author. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package validation
import (
"fmt"
"reflect"
"regexp"
"strconv"
"strings"
)
const (
// ValidTag struct tag
ValidTag = "valid"
LabelTag = "label"
wordsize = 32 << (^uint(0) >> 32 & 1)
)
var (
// key: function name
// value: the number of parameters
funcs = make(Funcs)
// doesn't belong to validation functions
unFuncs = map[string]bool{
"Clear": true,
"HasErrors": true,
"ErrorMap": true,
"Error": true,
"apply": true,
"Check": true,
"Valid": true,
"NoMatch": true,
}
// ErrInt64On32 show 32 bit platform not support int64
ErrInt64On32 = fmt.Errorf("not support int64 on 32-bit platform")
)
func init() {
v := &Validation{}
t := reflect.TypeOf(v)
for i := 0; i < t.NumMethod(); i++ {
m := t.Method(i)
if !unFuncs[m.Name] {
funcs[m.Name] = m.Func
}
}
}
// CustomFunc is for custom validate function
type CustomFunc func(v *Validation, obj interface{}, key string)
// AddCustomFunc Add a custom function to validation
// The name can not be:
//
// Clear
// HasErrors
// ErrorMap
// Error
// Check
// Valid
// NoMatch
//
// If the name is same with exists function, it will replace the origin valid function
func AddCustomFunc(name string, f CustomFunc) error {
if unFuncs[name] {
return fmt.Errorf("invalid function name: %s", name)
}
funcs[name] = reflect.ValueOf(f)
return nil
}
// ValidFunc Valid function type
type ValidFunc struct {
Name string
Params []interface{}
}
// Funcs Validate function map
type Funcs map[string]reflect.Value
// Call validate values with named type string
func (f Funcs) Call(name string, params ...interface{}) (result []reflect.Value, err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("%v", r)
}
}()
if _, ok := f[name]; !ok {
err = fmt.Errorf("%s does not exist", name)
return
}
if len(params) != f[name].Type().NumIn() {
err = fmt.Errorf("The number of params is not adapted")
return
}
in := make([]reflect.Value, len(params))
for k, param := range params {
in[k] = reflect.ValueOf(param)
}
result = f[name].Call(in)
return
}
func isStruct(t reflect.Type) bool {
return t.Kind() == reflect.Struct
}
func isStructPtr(t reflect.Type) bool {
return t.Kind() == reflect.Ptr && t.Elem().Kind() == reflect.Struct
}
func getValidFuncs(f reflect.StructField) (vfs []ValidFunc, err error) {
tag := f.Tag.Get(ValidTag)
label := f.Tag.Get(LabelTag)
if len(tag) == 0 {
return
}
if vfs, tag, err = getRegFuncs(tag, f.Name); err != nil {
return
}
fs := strings.Split(tag, ";")
for _, vfunc := range fs {
var vf ValidFunc
if len(vfunc) == 0 {
continue
}
vf, err = parseFunc(vfunc, f.Name, label)
if err != nil {
return
}
vfs = append(vfs, vf)
}
return
}
// Get Match function
// May be get NoMatch function in the future
func getRegFuncs(tag, key string) (vfs []ValidFunc, str string, err error) {
tag = strings.TrimSpace(tag)
index := strings.Index(tag, "Match(/")
if index == -1 {
str = tag
return
}
end := strings.LastIndex(tag, "/)")
if end < index {
err = fmt.Errorf("invalid Match function")
return
}
reg, err := regexp.Compile(tag[index+len("Match(/") : end])
if err != nil {
return
}
vfs = []ValidFunc{{"Match", []interface{}{reg, key + ".Match"}}}
str = strings.TrimSpace(tag[:index]) + strings.TrimSpace(tag[end+len("/)"):])
return
}
func parseFunc(vfunc, key string, label string) (v ValidFunc, err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("%v", r)
}
}()
vfunc = strings.TrimSpace(vfunc)
start := strings.Index(vfunc, "(")
var num int
// doesn't need parameter valid function
if start == -1 {
if num, err = numIn(vfunc); err != nil {
return
}
if num != 0 {
err = fmt.Errorf("%s require %d parameters", vfunc, num)
return
}
v = ValidFunc{vfunc, []interface{}{key + "." + vfunc + "." + label}}
return
}
end := strings.Index(vfunc, ")")
if end == -1 {
err = fmt.Errorf("invalid valid function")
return
}
name := strings.TrimSpace(vfunc[:start])
if num, err = numIn(name); err != nil {
return
}
params := strings.Split(vfunc[start+1:end], ",")
// the num of param must be equal
if num != len(params) {
err = fmt.Errorf("%s require %d parameters", name, num)
return
}
tParams, err := trim(name, key+"."+name+"."+label, params)
if err != nil {
return
}
v = ValidFunc{name, tParams}
return
}
func numIn(name string) (num int, err error) {
fn, ok := funcs[name]
if !ok {
err = fmt.Errorf("doesn't exists %s valid function", name)
return
}
// sub *Validation obj and key
num = fn.Type().NumIn() - 3
return
}
func trim(name, key string, s []string) (ts []interface{}, err error) {
ts = make([]interface{}, len(s), len(s)+1)
fn, ok := funcs[name]
if !ok {
err = fmt.Errorf("doesn't exists %s valid function", name)
return
}
for i := 0; i < len(s); i++ {
var param interface{}
// skip *Validation and obj params
if param, err = parseParam(fn.Type().In(i+2), strings.TrimSpace(s[i])); err != nil {
return
}
ts[i] = param
}
ts = append(ts, key)
return
}
// modify the parameters's type to adapt the function input parameters' type
func parseParam(t reflect.Type, s string) (i interface{}, err error) {
switch t.Kind() {
case reflect.Int:
i, err = strconv.Atoi(s)
case reflect.Int64:
if wordsize == 32 {
return nil, ErrInt64On32
}
i, err = strconv.ParseInt(s, 10, 64)
case reflect.Int32:
var v int64
v, err = strconv.ParseInt(s, 10, 32)
if err == nil {
i = int32(v)
}
case reflect.Int16:
var v int64
v, err = strconv.ParseInt(s, 10, 16)
if err == nil {
i = int16(v)
}
case reflect.Int8:
var v int64
v, err = strconv.ParseInt(s, 10, 8)
if err == nil {
i = int8(v)
}
case reflect.String:
i = s
case reflect.Ptr:
if t.Elem().String() != "regexp.Regexp" {
err = fmt.Errorf("not support %s", t.Elem().String())
return
}
i, err = regexp.Compile(s)
default:
err = fmt.Errorf("not support %s", t.Kind().String())
}
return
}
func mergeParam(v *Validation, obj interface{}, params []interface{}) []interface{} {
return append([]interface{}{v, obj}, params...)
}