Ignore binary data in config backup

[syn-bit]

• I have read and understood docs/Issues.md.

Describe the problem
My Netgear switch produced a line of binary data that changes each time a config dump is made. This means there is always a config change and it triggers a new config to be saved and a notification to be sent, even though there were no config changes.

So far I have deducted that the binary block always starts with \x82 and ends with \xB8 (some more bytes could be static, but this seems to be enough to recognize the line):

diff --git a/sw02 b/sw02
index 23352ad..565ec5c 100644
--- a/sw02
+++ b/sw02
@@ -20,7 +20,7 @@
 !
 !Additional Packages............................ FASTPATH QOS
 !                                                FASTPATH IPv6 Management
-!                                                <82>e<92>T<80><B0><D8>^\<81><E7><E6><B8>
+!                                                <82>e<C2><FC><80><B0><D8>^\<81><E7><E6><B8>
 !
 !!COMMAND: show bootvar
 !

I added a line like this to netgear.rb:

cfg.gsub! /(\x82.*\xB8)/, '<removed>'

And added # encoding: US-ASCII at the top to fix the error invalid multibyte escape: /(\\x82.*)/ (SyntaxError)

However, this does not seem to work.

I have very little experience with Ruby, so I hope someone can help me write the right line to ignore changes like above when checking for config changes.

Expected behavior
The binary data being ignored in config backups

Configuration
See above

Logs
N/A

Running environment (please complete the following information):

• OS: Debian Bookworm
• oxidized version: 0.30.1

Additional context
N/A

[ytti]

You have few options.

string.encode("UTF-8", invalid: replace, undef: replace, replace: '?')
string.b

The first returns the string asUTF-8 replacing non-UTF-8 with ?. The second returns binary string, which is basically encode-less 8bit clean.

[ytti]

irb(main):001:0> "moi".sub /#{"(\x82.*\xB8)".b}/, ''
=> "moi"

Bit less ugly version

irb(main):001> re = Regexp.new "(\x82.*\xB8)".b
=> /(\x82.*\xB8)/
irb(main):002> "moi".match re
=> nil

The problem is that the regex string itself is assumed to be UTF-8, and what you're inputting is not. I'm not sure if either is right way to do it.

EDIT. I think this is the right way to do it:

irb(main):001> "moi".match /(\x82.*\xB8)/n
=> nil

https://ruby-doc.org/core-3.1.1/Regexp.html

[Encoding](https://ruby-doc.org/core-3.1.1/Encoding.html)[¶](https://ruby-doc.org/core-3.1.1/Regexp.html#class-Regexp-label-Encoding) [↑](https://ruby-doc.org/core-3.1.1/Regexp.html#top)

Regular expressions are assumed to use the source encoding. This can be overridden with one of the following modifiers.

/pat/u - UTF-8
/pat/e - EUC-JP
/pat/s - Windows-31J
/pat/n - ASCII-8BIT

[syn-bit]

I got it to work, based on the first part of your comment (funny, the mail notification shows only part of a comment and I did not know that, so did not see the rest of your comment when started experimenting, based on your comment).

s = "214164646974696f6e616c205061636b616765732e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e2e20464153545041544820514f530a2120202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202046415354504154482049507636204d616e6167656d656e740a212020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020208265c2fc80b0d81c81e7e6b80a210a2121434f4d4d414e443a2073686f7720626f6f747661720a"
s = s.scan(/../).map { |x| x.hex.chr }.join
s.gsub! /#{"(\x82.*\xB8)".b}/, '<binary data removed>'
print(s)

Also, I put the gsub in the wrong part (config instead of version).

I now have:

  cmd 'show version' do |cfg|
    cfg.gsub! /(Current Time\.+ ).*/, '\\1 <removed>'
    cfg.gsub! /#{"(\x82.*\xB8.*$)".b}/, '<binary data removed>'
    comment cfg
  end

I'm not sure how many people have issues with the binary data, should I create a PR for it?