Skip to content

Add EnzymeRules for batchnorm#537

Open
wsmoses wants to merge 15 commits into
FluxML:masterfrom
wsmoses:batchnorm
Open

Add EnzymeRules for batchnorm#537
wsmoses wants to merge 15 commits into
FluxML:masterfrom
wsmoses:batchnorm

Conversation

@wsmoses

@wsmoses wsmoses commented Sep 25, 2023

Copy link
Copy Markdown
Contributor

Follow up to #536

I made this a separate PR, because for efficiency sake, it is better to have an inplace version of batchnorm, but that wasn't an exported API. Thus I implemented this on the inplace cuda one (which was the only implementation of batchnorm, in place or not).

If NNlib is later refactored to have batchnorm! this can go on the refactored rule.

@mashu

mashu commented Sep 25, 2023

Copy link
Copy Markdown
Contributor

Few suggestions ;)

  1. syntax: more than one semicolon in argument list around around line batchnorm.jl:288
  2. refactor into single line all || in if statements as starting new line with || fails compilation on 1.9.3

@CarloLucibello

Copy link
Copy Markdown
Member

There is a weird parsing error in the Code Coverage stage

@CarloLucibello

CarloLucibello commented Nov 7, 2023

Copy link
Copy Markdown
Member

Does anyone know about the ubuntu CI error

ERROR: LoadError: Base.Meta.ParseError("parsing error in ext/NNlibCUDACUDNNExt/batchnorm.jl:253: invalid identifier name \"||\"")
Stacktrace:
 [1] amend_coverage_from_src!(fc::FileCoverage)
   @ CoverageTools ~/.julia/packages/CoverageTools/JSYpL/src/CoverageTools.jl:186

?

@mashu

mashu commented Nov 7, 2023

Copy link
Copy Markdown
Contributor

Does anyone know about the ubuntu CI error

ERROR: LoadError: Base.Meta.ParseError("parsing error in ext/NNlibCUDACUDNNExt/batchnorm.jl:253: invalid identifier name \"||\"")
Stacktrace:
 [1] amend_coverage_from_src!(fc::FileCoverage)
   @ CoverageTools ~/.julia/packages/CoverageTools/JSYpL/src/CoverageTools.jl:186

?

I couldn't compile the branch on my local machine with Julia 1.9.3. I was also running into problems with parser not figuring out it's the new line, see my comment earlier. Basically code won't compile for me without fixing these.

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants