Mercurial > hg > nginx-site
changeset 2576:4c8d0b37932d
Corrected syntax and style of "Using node modules with njs".
| author | Yaroslav Zhuravlev <yar@nginx.com> |
|---|---|
| date | Thu, 06 Aug 2020 14:46:58 +0100 |
| parents | 2839ad72d8ef |
| children | 67fd664e2612 |
| files | xml/en/docs/njs/node_modules.xml |
| diffstat | 1 files changed, 68 insertions(+), 97 deletions(-) [+] |
line wrap: on
line diff
--- a/xml/en/docs/njs/node_modules.xml +++ b/xml/en/docs/njs/node_modules.xml @@ -9,20 +9,19 @@ <article name="Using node modules with njs" link="/en/docs/njs/node_modules.html" lang="en" - rev="3"> + rev="4"> -<section id="intro" name="Introduction"> +<section id="intro"> <para> -Often, a developer wants to use 3rd-party code, usually available as a library -of some kind. -In the Javascript world, the concept of a module is relatively new, so there -was no standard until recently. +Often, a developer wants to use 3rd-party code, +usually available as a library of some kind. +In the JavaScript world, the concept of a module is relatively new, +so there was no standard until recently. Many platforms (browsers) still don't support modules, which makes code reuse harder. -The njs does not (yet) support modules, too. -This article describes ways to overcome this limitation, using the -<link url="https://nodejs.org/">Node.js</link> ecosystem as an example. +This article describes ways to reuse +<link url="https://nodejs.org/">Node.js</link> code in njs. </para> <note> @@ -32,30 +31,32 @@ Examples in this article use features th </note> <para> -There is a number of issues that may arise when 3rd-party code is added to njs: +There is a number of issues +that may arise when 3rd-party code is added to njs: <list type="bullet"> -<listitem>Multiple files that reference each other, and their -dependencies</listitem> +<listitem> +Multiple files that reference each other and their dependencies +</listitem> -<listitem>Platform-specific APIs</listitem> +<listitem> +Platform-specific APIs +</listitem> -<listitem>Modern standard language constructions</listitem> +<listitem> +Modern standard language constructions +</listitem> </list> - </para> <para> -The good news is that such problems are not something new or -specific to njs. -Javascript developers face them daily when trying to support multiple -disparate platforms with very different properties. +The good news is that such problems are not something new or specific to njs. +JavaScript developers face them daily +when trying to support multiple disparate platforms +with very different properties. There are instruments designed to resolve the above-mentioned issues. -</para> - -<para> <list type="bullet"> @@ -74,8 +75,8 @@ your code and all the dependencies. <listitem> Platform-specific APIs <para> -You can use multiple libraries that implement such APIs in a platform-agnostic -manner (at the expense of performance, though). +You can use multiple libraries that implement such APIs +in a platform-agnostic manner (at the expense of performance, though). Particular features can also be implemented using the <link url="https://polyfill.io/v3/">polyfill</link> approach. </para> @@ -84,57 +85,52 @@ Particular features can also be implemen <listitem> Modern standard language constructions <para> -Such code can be transpiled: this means performing a number of transformations +Such code can be transpiled: +this means performing a number of transformations that rewrite newer language features in accordance with an older standard. -For example, <link url="https://babeljs.io/"> babel</link> project can -be used to this purpose. +For example, <link url="https://babeljs.io/"> babel</link> project +can be used to this purpose. </para> </listitem> </list> - </para> - <para> In this guide, we will use two relatively large npm-hosted libraries: <list type="bullet"> <listitem> -<link url="https://www.npmjs.com/package/protobufjs">protobufjs</link> - +<link url="https://www.npmjs.com/package/protobufjs">protobufjs</link>— a library for creating and parsing protobuf messages used by the -<link url="https://grpc.io/">gRPC</link> protocol. - +<link url="https://grpc.io/">gRPC</link> protocol </listitem> <listitem> -<link url="https://www.npmjs.com/package/dns-packet">dns-packet</link> - -a library for processing DNS protocol packets. +<link url="https://www.npmjs.com/package/dns-packet">dns-packet</link>— +a library for processing DNS protocol packets </listitem> </list> - </para> </section> + <section id="environment" name="Environment"> <para> - <note> -This document mostly employs a generic approach and AVOIDS specific best -practice advices concerning Node.js and the rapidly evolving JavaScript -ecosystem. -Make sure to consult the corresponding package's manual BEFORE following the -steps suggested here. +This document mostly employs a generic approach +and avoids specific best practice advices concerning Node.js +and JavaScript. +Make sure to consult the corresponding package's manual +before following the steps suggested here. </note> - First (assuming Node.js is installed and operational), let's create an -empty project and install some dependencies; the commands below assume we're -in the working directory: - +empty project and install some dependencies; +the commands below assume we are in the working directory: <example> $ mkdir my_project && cd my_project $ npx license choose_your_license_here > LICENSE @@ -162,44 +158,47 @@ EOF </section> + <section id="protobuf" name="Protobufjs"> <para> -The library provides a parser for the <literal>.proto</literal> interface -definitions and a code generator for message parsing and generation. +The library provides a parser +for the <literal>.proto</literal> interface definitions +and a code generator for message parsing and generation. </para> <para> In this example, we will use the <link url="https://github.com/grpc/grpc/blob/master/examples/protos/helloworld.proto">helloworld.proto</link> -file from the gRPC examples. -Our goal is to create two messages: <literal>HelloRequest</literal> and +file +from the gRPC examples. +Our goal is to create two messages: +<literal>HelloRequest</literal> and <literal>HelloResponse</literal>. We will use the <link url="https://github.com/protobufjs/protobuf.js/blob/master/README.md#reflection-vs-static-code">static</link> mode of protobufjs instead of dynamically generating classes, because -njs doesn't support adding new functions dynamically due to security -considerations. +njs doesn't support adding new functions dynamically +due to security considerations. </para> <para> -Next, the library is installed and javascript code implementing -message marshalling is generated from the protocol definition: +Next, the library is installed and +the JavaScript code implementing message marshalling +is generated from the protocol definition: <example> $ npm install protobufjs $ npx pbjs -t static-module helloworld.proto > static.js </example> - </para> <para> -Thus, the <literal>static.js</literal> file becomes our new dependency, storing -all the code we need to implement message processing. +Thus, the <literal>static.js</literal> file becomes our new dependency, +storing all the code we need to implement message processing. The <literal>set_buffer()</literal> function contains code that uses the -library to create a buffer with the serialized <literal>HelloRequest</literal> -message. +library to create a buffer with the serialized +<literal>HelloRequest</literal> message. The code resides in the <literal>code.js</literal> file: - <example> var pb = require('./static.js'); @@ -220,7 +219,7 @@ function set_buffer(pb) var frame = new Uint8Array(5 + buffer.length); frame[0] = 0; // 'compressed' flag - frame[1] = (n & 0xFF000000) >>> 24; // length: uint32 in network order + frame[1] = (n & 0xFF000000) >>> 24; // length: uint32 in network byte order frame[2] = (n & 0x00FF0000) >>> 16; frame[3] = (n & 0x0000FF00) >>> 8; frame[4] = (n & 0x000000FF) >>> 0; @@ -236,7 +235,6 @@ var frame = set_buffer(pb); <para> To ensure it works, we execute the code using node: - <example> $ node ./code.js Uint8Array [ @@ -245,10 +243,8 @@ Uint8Array [ 116, 114, 105, 110, 103 ] </example> - You can see that this got us a properly encoded <literal>gRPC</literal> frame. Now let's run it with njs: - <example> $ njs ./code.js Thrown: @@ -266,18 +262,17 @@ or other similar tool. <para> An attempt to process our existing <literal>code.js</literal> file will result -in a bunch of JS code that is supposed to run in a browser, i.e. immediately -upon loading. +in a bunch of JS code that is supposed to run in a browser, +i.e. immediately upon loading. This isn't something we actually want. Instead, we want to have an exported function that can be referenced from the nginx configuration. This requires some wrapper code. - <note> -In this guide, we use njs cli in all examples for the sake of simplicity. +In this guide, we use +njs <link doc="cli.xml">cli</link> in all examples for the sake of simplicity. In real life, you will be using nginx njs module to run your code. </note> - </para> <para> @@ -292,13 +287,12 @@ the library. </para> <para> -Next, we process it with browserify to get all dependencies into a single file: +Next, we process it with <literal>browserify</literal> +to get all dependencies into a single file: <example> $ npx browserify load.js -o bundle.js -d </example> - -The result is huge file that contains all our dependencies: - +The result is a huge file that contains all our dependencies: <example> (function(){function...... ... @@ -306,10 +300,8 @@ The result is huge file that contains al },{"protobufjs/minimal":9}]},{},[1]) //# sourceMappingURL.............. </example> - To get final "<literal>njs_bundle.js</literal>" file we concatenate "<literal>bundle.js</literal>" and the following code: - <example> // Example usage of protobuf library: prepare a buffer to send function set_buffer(pb) @@ -328,7 +320,7 @@ function set_buffer(pb) var frame = new Uint8Array(5 + buffer.length); frame[0] = 0; // 'compressed' flag - frame[1] = (n & 0xFF000000) >>> 24; // length: uint32 in network order + frame[1] = (n & 0xFF000000) >>> 24; // length: uint32 in network byte order frame[2] = (n & 0x00FF0000) >>> 16; frame[3] = (n & 0x0000FF00) >>> 8; frame[4] = (n & 0x000000FF) >>> 0; @@ -348,7 +340,6 @@ function setbuf() var frame = setbuf(); console.log(frame); </example> - Let's run the file using node to make sure things still work: <example> $ node ./njs_bundle.js @@ -358,14 +349,11 @@ Uint8Array [ 116, 114, 105, 110, 103 ] </example> - Now let's proceed further with njs: - <example> $ /njs ./njs_bundle.js Uint8Array [0,0,0,0,12,10,10,84,101,115,116,83,116,114,105,110,103] </example> - The last thing will be to use njs-specific API to convert array into byte string, so it could be usable by nginx module. We can add the following snippet before the last line: @@ -374,9 +362,7 @@ if (global.njs) { return String.bytesFrom(frame) } </example> - Finally, we got it working: - <example> $ njs ./njs_bundle.js |hexdump -C 00000000 00 00 00 00 0c 0a 0a 54 65 73 74 53 74 72 69 6e |.......TestStrin| @@ -418,6 +404,7 @@ function parse_msg(pb, msg) </section> + <section id="dnspacket" name="DNS-packet"> <para> @@ -435,7 +422,6 @@ Additional node packages are needed: $ npm install buffer $ npm install dns-packet </example> - The configuration file, webpack.config.js: <example> const path = require('path'); @@ -470,31 +456,23 @@ module.exports = { Note we are using "<literal>production</literal>" mode. In this mode webpack does not use "<literal>eval</literal>" construction not supported by njs. - The referenced <literal>load.js</literal> file is our entry point: <example> global.dns = require('dns-packet') global.Buffer = require('buffer/').Buffer </example> - We start the same way, by producing a single file for the libraries: - <example> $ npx browserify load.js -o bundle.js -d </example> - Next, we process the file with webpack, which itself invokes babel: - <example> $ npx webpack --config webpack.config.js </example> - This command produces the <literal>dist/wp_out.js</literal> file, which is a transpiled version of <literal>bundle.js</literal>. - We need to concatenate it with <literal>code.js</literal> that stores our code: - <example> function set_buffer(dnsPacket) { @@ -512,23 +490,18 @@ function set_buffer(dnsPacket) return buf; } </example> - Note that in this example generated code is not wrapped into function and we do not need to call it explicitly. The result is in the "<literal>dist</literal>" directory: - <example> $ cat dist/wp_out.js code.js > njs_dns_bundle.js </example> - Let's call our code at the end of a file: <example> var b = setbuf(1); console.log(b); </example> - And execute it using node: - <example> $ node ./njs_dns_bundle_final.js Buffer [Uint8Array] [ @@ -538,7 +511,6 @@ Buffer [Uint8Array] [ 0, 1, 0, 1 ] </example> - Make sure this works as expected, and then run it with njs: <example> $ njs ./njs_dns_bundle_final.js @@ -563,7 +535,6 @@ function parse_response(buf) // expected name is 'google.com', according to our request above } </example> - </para> </section>