Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ if the initial content request or any subsequent asset request returns a bad res
The `raise_on_js_error` option, when enabled, will raise a `Grover::JavaScript::PageRenderError` if any uncaught
Javascript errors occur when trying to render the page.

The `javascript_enabled` option can be used to disable JavaScript. See https://github.com/puppeteer/puppeteer/blob/main/docs/api/puppeteer.page.setjavascriptenabled.md

The Chrome/Chromium executable path can be overridden with the `executable_path` option.

Supplementary JavaScript can be executed on the page (after render and before conversion to PDF/image)
Expand Down
5 changes: 5 additions & 0 deletions lib/grover/js/processor.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,11 @@ const _processPage = (async (convertAction, uriOrHtml, options) => {
await cdp.send('Browser.setPermission', { permission: { name: 'local-network-access' }, setting: 'granted' });
}

const javaScriptEnabled = options.javaScriptEnabled; delete options.javaScriptEnabled;
if (javaScriptEnabled !== undefined) {
await page.setJavaScriptEnabled(javaScriptEnabled);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the feature, but this does feel a little naive w.r.t the other options that explicitly require JS. Specifically:

  • executeScript
  • evaluateOnNewDocument
  • waitForFunction
  • scriptTagOptions

I see four options:

  1. we ignore any misconfigurations (combination of JS disabled and any of the above options provided) and let the failure ensue
  2. we implicitly disable the above three options is JS is disabled (probably just as extra guards in processor.cjs)
  3. we raise warnings if JS is disabled (probably in OptionsFixer) and any of the above options are provided, but still let the failure happen
  4. we raise warnings if JS is disabled (probably in OptionsFixer) and any of the above options are provided, but disable said options (probably just as extra guards in processor.cjs)

My preference would be (in order) options 4, 2, 3, then 1.. but very much preferencing option 4.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To demonstrate the behaviour:

Grover.new(
  <<~HTML,
    <div>Static content</div>
    <script>doneProcessing = true</script>
  HTML
  javascript_enabled: false,
  wait_for_function: 'doneProcessing === true'
).to_pdf('foo.pdf'); nil

This will time out, because even though we set doneProcessing, it never gets called.. but Grover will still try wait for it to be there.

And if you hard code the wait_for_function to just "true", it'll return.. so even with JS "disabled", the page will still actually be running JS.. sort of.

If we were to take out the wait_for_function bit, you might expect that the page would come back only with the static content..

Grover.new(
  <<~HTML,
    <div>Static content</div>
    <noscript>No script!</noscript>
    <div id="foo"></div>
    <div id="bar"></div>
    <div id="baz"></div>
    <div id="qux"></div>
    <script>document.getElementById("foo").innerHTML="this is foo"</script>
  HTML
  javascript_enabled: false,
  execute_script: "document.getElementById('bar').innerHTML='this is bar'",
  evaluate_on_new_document: "document.getElementById('baz').innerHTML='this is baz'",
  script_tag_options: [{ 'content' => "document.getElementById('qux').innerHTML='this is qux'" }]
).to_pdf('foo.pdf'); nil

but actually you get:

Image

So it isn't enabled.. great.. but you can still execute script.. doh.

I know these are really Puppeteer issues, and or could be blamed on the end user for.. poor life (prop) choices 😆, but we know better.. and can do something about it 😄

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 31f241d the JavaScript options are now disabled when javascript_enabled is false.

}

const raiseOnRequestFailure = options.raiseOnRequestFailure; delete options.raiseOnRequestFailure;
if (raiseOnRequestFailure) {
page.on('requestfinished', (request) => {
Expand Down
21 changes: 20 additions & 1 deletion lib/grover/options_fixer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ class Grover
class OptionsFixer
FALSE_VALUES = [nil, false, 0, '0', 'f', 'F', 'false', 'FALSE', 'off', 'OFF'].freeze

JAVASCRIPT_OPTIONS = %w[evaluate_on_new_document execute_script script_tag_options wait_for_function].freeze
private_constant :JAVASCRIPT_OPTIONS

def initialize(options)
@options = options
end
Expand All @@ -18,6 +21,7 @@ def run
fix_integer_options!
fix_float_options!
fix_array_options!
disable_javascript_options!
@options
end

Expand All @@ -35,7 +39,7 @@ def fix_boolean_options!
fix_options!(
'display_header_footer', 'full_page', 'landscape', 'omit_background', 'prefer_css_page_size',
'print_background', 'viewport.has_touch', 'viewport.is_landscape', 'viewport.is_mobile', 'bypass_csp',
'raise_on_request_failure', 'raise_on_js_error'
'raise_on_request_failure', 'raise_on_js_error', 'javascript_enabled'
) { |value| !FALSE_VALUES.include?(value) }
end

Expand All @@ -60,5 +64,20 @@ def fix_array_options!
value.is_a?(String) ? YAML.safe_load(value) : value
end
end

def disable_javascript_options!
return if @options['javascript_enabled'] != false

JAVASCRIPT_OPTIONS.each do |option|
disable_javascript_option!(option)
end
end

def disable_javascript_option!(option)
return unless @options.key?(option)

@options.delete(option)
warn "#{self.class}: option #{option} has been disabled because javascript_enabled is set to false"
end
end
end
7 changes: 4 additions & 3 deletions lib/grover/utils.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ class Utils
'css' => 'CSS',
'csp' => 'CSP',
'http' => 'HTTP',
'js' => 'JS'
'js' => 'JS',
'javascript' => 'JavaScript'
}.freeze
private_constant :ACRONYMS

Expand Down Expand Up @@ -94,9 +95,9 @@ def self.normalize_object(object, excluding: [])
# Regex sourced from ActiveSupport camelize
#
def self.normalize_key(key)
key.to_s.downcase.gsub(%r{(?:_|(/))([a-z\d]*)}) do
key.to_s.downcase.gsub(%r{(?:^|_|(/))([a-z\d]*)}) do
"#{Regexp.last_match(1)}#{ACRONYMS[Regexp.last_match(2)] || Regexp.last_match(2).capitalize}"
end
end.sub(/^[[:alpha:]]/, &:downcase)
end
private_class_method :normalize_key
end
Expand Down
36 changes: 36 additions & 0 deletions spec/grover/options_fixer_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -109,4 +109,40 @@
expect(fixed_options['launch_args']).to eq ['--some-option']
end
end

context 'when javascript_enabled option is set' do
let(:options) do
{
'javascript_enabled' => enabled,
'evaluate_on_new_document' => 'test 1',
'execute_script' => 'test 2',
'script_tag_options' => 'test 3',
'wait_for_function' => 'test 4'
}
end

context 'when it is true' do
let(:enabled) { 'true' }

it 'keep javascript options' do
expect(fixed_options['javascript_enabled']).to be true
expect(fixed_options['evaluate_on_new_document']).to eq 'test 1'
expect(fixed_options['execute_script']).to eq 'test 2'
expect(fixed_options['script_tag_options']).to eq 'test 3'
expect(fixed_options['wait_for_function']).to eq 'test 4'
end
end

context 'when it is false' do
let(:enabled) { 'false' }

it 'disable javascript options' do
expect(fixed_options['javascript_enabled']).to be false
expect(fixed_options['evaluate_on_new_document']).to be_nil
expect(fixed_options['execute_script']).to be_nil
expect(fixed_options['script_tag_options']).to be_nil
expect(fixed_options['wait_for_function']).to be_nil
end
end
end
end
29 changes: 29 additions & 0 deletions spec/grover/processor_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -1190,6 +1190,35 @@
end
end

context 'when javaScriptEnabled option is set' do
let(:options) { basic_header_footer_options.merge('javaScriptEnabled' => enabled) }

let(:url_or_html) do
<<-HTML
<html>
<body>
<script>document.write('output from script tag')</script>
<noscript>output from noscript tag</noscript>
</body>
</html>
HTML
end

context 'when it is true' do
let(:enabled) { true }

it { expect(pdf_text_content).to include 'output from script tag' }
it { expect(pdf_text_content).not_to include 'output from noscript tag' }
end

context 'when it is false' do
let(:enabled) { false }

it { expect(pdf_text_content).not_to include 'output from script tag' }
it { expect(pdf_text_content).to include 'output from noscript tag' }
end
end

# < v23 of puppeteer has issues installing the latest firefox versions (bz2 vs xz compression used for packaging)
if puppeteer_version_on_or_after? '23'
context 'when specifying Firefox browser' do
Expand Down
9 changes: 6 additions & 3 deletions spec/grover/utils_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,8 @@
prefer_css_page_size: true,
bypass_csp: false,
extra_http_headers: { 'Foo' => 'Bar' },
raise_on_js_error: true
raise_on_js_error: true,
javascript_enabled: false
}
end

Expand All @@ -314,7 +315,8 @@
'preferCSSPageSize' => true,
'bypassCSP' => false,
'extraHTTPHeaders' => { 'foo' => 'Bar' },
'raiseOnJSError' => true
'raiseOnJSError' => true,
'javaScriptEnabled' => false
)
)
end
Expand All @@ -328,7 +330,8 @@
'preferCSSPageSize' => true,
'bypassCSP' => false,
'extraHTTPHeaders' => { 'Foo' => 'Bar' },
'raiseOnJSError' => true
'raiseOnJSError' => true,
'javaScriptEnabled' => false
)
)
end
Expand Down
Loading