Skip to content

feat: add urlencode and urldecode modifiers for string manipulation #699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 21 additions & 6 deletions docs/howitworks.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,14 +92,14 @@ metadata:
**Note**: This ignored for secret managers that don't allow versioning, meaning the latest version is returned

#### Inline-path placeholders
An inline-path placeholder allows you to specify the path, key, and optionally, the version to use for a specific placeholder. This means you can inject values from _multiple distinct_ secrets in your secrets manager into the same YAML.
An inline-path placeholder allows you to specify the path, key, and optionally, the version to use for a specific placeholder. This means you can inject values from _multiple distinct_ secrets in your secrets manager into the same YAML.

Valid examples:

- `<path:some/path#secret-key>`
- `<path:some/path#secret-key#version>`

If the `version` is omitted (first example), the latest version of the secret is retrieved.
If the `version` is omitted (first example), the latest version of the secret is retrieved.

##### Specifying the path of a secret
The only way to specify the path is in the placeholder itself: the string `path:` followed by the path in your secret manager to the secret. The `avp.kubernetes.io/path` annotation has _no effect_ on these placeholders.
Expand All @@ -112,9 +112,9 @@ The only way to specify the version is in the placeholder itself: the string fol
#### Special behavior

##### Base64 placeholders
Some tools like Kustomize secret generator will create Secrets with `data` fields containing base64 encoded strings from the source files. If you try to use `<placeholder>`s in the source files, they will be output in a base64 format.
Some tools like Kustomize secret generator will create Secrets with `data` fields containing base64 encoded strings from the source files. If you try to use `<placeholder>`s in the source files, they will be output in a base64 format.

The plugin can handle this case by finding any base64 encoded placeholders (either generic or inline-path), replace them, and re-base64 encode the result.
The plugin can handle this case by finding any base64 encoded placeholders (either generic or inline-path), replace them, and re-base64 encode the result.

For example, given this input:
```yaml
Expand Down Expand Up @@ -211,9 +211,9 @@ fieldRef:
```

##### Removing keys with missing values
By default, AVP will return an error if there is a `<placeholder>` that has no matching key in the secrets manager.
By default, AVP will return an error if there is a `<placeholder>` that has no matching key in the secrets manager.

You can override this by using the annotation `avp.kubernetes.io/remove-missing`. This will remove keys whose values are missing from Vault from the entire YAML.
You can override this by using the annotation `avp.kubernetes.io/remove-missing`. This will remove keys whose values are missing from Vault from the entire YAML.

For example, given this input:
```yaml
Expand Down Expand Up @@ -338,6 +338,21 @@ spec:
checksum/secret: <path:secrets/data/db#certs | sha256sum>
```

##### `urlencode` / `urldecode`

The urlencode and urldecode modifiers are used to encode a string into a format that can be safely included in a URL

Valid examples:

```yaml
kind: Deployment
spec:
template:
metadata:
annotations:
checksum/secret: <path:secrets/data/db#certs | urlencode>
```

### Error Handling

#### Detecting errors in chained commands
Expand Down
32 changes: 32 additions & 0 deletions pkg/kube/modifiers.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,38 @@ var modifiers = map[string]func([]string, interface{}) (interface{}, error){
"yamlParse": yamlParse,
"indent": indent,
"sha256sum": sha256sum,
"urldecode": urldecode,
"urlencode": urlencode,
}

func urlencode(params []string, input interface{}) (interface{}, error) {
if len(params) > 0 {
return nil, fmt.Errorf("invalid parameters")
}
switch input.(type) {
case string:
{
s := strings.ReplaceAll(input.(string), " ", "%20")
return s, nil
}
default:
return nil, fmt.Errorf("invalid datatype %v, expected string", reflect.TypeOf(input))
}
}

func urldecode(params []string, input interface{}) (interface{}, error) {
if len(params) > 0 {
return nil, fmt.Errorf("invalid parameters")
}
switch input.(type) {
case string:
{
s := strings.ReplaceAll(input.(string), "%20", " ")
return s, nil
}
default:
return nil, fmt.Errorf("invalid datatype %v, expected string", reflect.TypeOf(input))
}
}

func indent(params []string, input interface{}) (interface{}, error) {
Expand Down
52 changes: 52 additions & 0 deletions pkg/kube/modifiers_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -245,3 +245,55 @@ func TestSha256Sum_success(t *testing.T) {
assertErrorEqual(t, nil, err)
assertResultEqual(t, expected, res)
}

func TestUrlEncode_invalidParams(t *testing.T) {
var data interface{} = "mysecret"
expectedErr := fmt.Errorf("invalid parameters")
_, err := urlencode([]string{"astring"}, data)
assertErrorEqual(t, expectedErr, err)
}

func TestUrlEncode_invalidDataType(t *testing.T) {
var data interface{} = map[string]interface{}{
"data": map[string]interface{}{
"subkey": "secret",
},
}
expectedErr := fmt.Errorf("invalid datatype map[string]interface {}, expected string")
_, err := urlencode([]string{}, data)
assertErrorEqual(t, expectedErr, err)
}

func TestUrlEncode_success(t *testing.T) {
var data interface{} = "my secret"
var expected interface{} = "my%20secret"
res, err := urlencode([]string{}, data)
assertErrorEqual(t, nil, err)
assertResultEqual(t, expected, res)
}

func TestUrlDecode_invalidParams(t *testing.T) {
var data interface{} = "my%20secret"
expectedErr := fmt.Errorf("invalid parameters")
_, err := urldecode([]string{"astring"}, data)
assertErrorEqual(t, expectedErr, err)
}

func TestUrlDecode_invalidDataType(t *testing.T) {
var data interface{} = map[string]interface{}{
"data": map[string]interface{}{
"subkey": "secret",
},
}
expectedErr := fmt.Errorf("invalid datatype map[string]interface {}, expected string")
_, err := urldecode([]string{}, data)
assertErrorEqual(t, expectedErr, err)
}

func TestUrlDecode_success(t *testing.T) {
var data interface{} = "my%20secret"
var expected interface{} = "my secret"
res, err := urldecode([]string{}, data)
assertErrorEqual(t, nil, err)
assertResultEqual(t, expected, res)
}