Mdn Web Development

Posted on 30-04-2021 by admin

QUESTIONS ABOUT MODULAR DEVELOPMENT? WE CAN HELP Contact Us MDN Development, LLC 876 Roser Park Drive South St. Petersburg, FL 33701 Call 813-376-9522 Get in touch. In this video, we take a talk about the Mozilla Developer Network or MDN for short. The Mozilla Developer Network (MDN) provides information about Open Web t.

Learn Web Development Mdn
Mdm Web Docs
Mdn Learn Web Development
Mozilla Web Dev
Mdn Web Development Software

This article provides information about the changes in Firefox 88 that will affect developers. Firefox 88 was released on April 19, 2021.

Note

See also Never too late for Firefox 88 on Mozilla Hacks.

Changes for web developers

Developer Tools

The toggle button for switching between raw and formatted response views has been implemented (bug 1693147). For examples, see Network request details > Response tab.

HTML

CSS

The :user-valid and :user-invalid pseudo-classes have been implemented (bug 1694141).
The image-set() functional notation is now implemented for content and cursor (bug 1695402 and bug 1695403).
The image-set() functional notation is now enabled (bug 1698133).
The default monospace font for MacOS has been changed to Menlo (bug 1342741).
The collapse value of visibility is now implemented for ruby annotations (bug 1697529).
The alternate value for ruby-position has been implemented, and is the new initial value for the property (bug 1694748).
The outline CSS property has been updated to follow the outline created by border-radius. As part of this work the non-standard -moz-outline-radius property has been removed. (bug 315209 and bug 1694146.)

Removals

The :-moz-submit-invalid pseudo-class has been hidden behind a preference, thereby removing it from web content (bug 1694129).
Default styling for the non-standard :user-invalid and :-moz-ui-valid has been removed (bug 1693969).

JavaScript

Added support for RegExp match indices (bug 1519483).
Intl.DisplayNames() and Intl.ListFormat() now have stricter checking that options passed to the constructor are objects, and will throw an exception if a string or other primitive is used instead (bug 1696881).

HTTP

FTP has been disabled on all releases (preference network.ftp.enabled now defaults to false), with the intent of removing it altogether in Firefox 90 (bug 1691890). Complementing this change, the extension setting browserSettings.ftpProtocolEnabled has been made read-only, and web extensions can now register themselves as protocol handlers for FTP (bug 1626365).

Security

APIs

DOM

Code can now use the new static method AbortSignal.abort() to return an AbortSignal that is already set as aborted (bug 1698468).

WebDriver conformance (Marionette)

Marionette will no longer be enabled unless the --marionette command line argument or the MOZ_MARIONETTE environment variable is specified. As such the marionette.enabled preference is no longer used. With this change the state of navigator.webdriver now correctly reflects the enabled state of Marionette (bug 1593343).
Fixed a bug where pointer actions other than down and up inappropriately led to buttons being pressed (bug 1686361).
Fixed a race condition in WebDriver:GetCurrentURL that could have led the command to return the URL of the previously opened page, or even a hang in Marionette (bug 1664881).

Changes for add-on developers

url can now be used to limit the properties for which the tabs.onUpdated event is triggered (bug 1680279).

Older versions

tl;dr: Periodically, the whole of MDN is built, by our Node code, in a GitHub Action. A Python script bulk-publishes this to Elasticsearch. Our Django server queries the same Elasticsearch via /api/v1/search. The site-search page is a static single-page app that sends XHR requests to the /api/v1/search endpoint. Search results’ sort-order is determined by match and “popularity”.

Jamstack’ing

The challenge with “Jamstack” websites is with data that is too vast and dynamic that it doesn’t make sense to build statically. Search is one of those. For the record, as of Feb 2021, MDN consists of 11,619 documents (aka. articles) in English. Roughly another 40,000 translated documents. In English alone, there are 5.3 million words. So to build a good search experience we need to, as a static site build side-effect, index all of this in a full-text search database. And Elasticsearch is one such database and it’s good. In particular, Elasticsearch is something MDN is already quite familiar with because it’s what was used from within the Django app when MDN was a wiki.

Note: MDN gets about 20k site-searches per day from within the site.

Build

When we build the whole site, it’s a script that basically loops over all the raw content, applies macros and fixes, dumps one index.html (via React server-side rendering) and one index.json. The index.json contains all the fully rendered text (as HTML!) in blocks of “prose”. It looks something like this:

You can see one here: /en-US/docs/Web/index.json

Indexing

Next, after all the index.json files have been produced, a Python script takes over and it traverses all the index.json files and based on that structure it figures out the, title, summary, and the whole body (as HTML).

Next up, before sending this into the bulk-publisher in Elasticsearch it strips the HTML. It’s a bit more than just turning <p>Some <em>cool</em> text.</p> to Some cool text. because it also cleans up things like <div> and certain <div> blocks.

One thing worth noting is that this whole thing runs roughly every 24 hours and then it builds everything. But what if, between two runs, a certain page has been removed (or moved), how do you remove what was previously added to Elasticsearch? The solution is simple: it deletes and re-creates the index from scratch every day. The whole bulk-publish takes a while so right after the index has been deleted, the searches won’t be that great. Someone could be unlucky in that they’re searching MDN a couple of seconds after the index was deleted and now waiting for it to build up again.
It’s an unfortunate reality but it’s a risk worth taking for the sake of simplicity. Also, most people are searching for things in English and specifically the Web/ tree so the bulk-publishing is done in a way the most popular content is bulk-published first and the rest was done after. Here’s what the build output logs:

So, yes, for 3m 35s there’s stuff missing from the index and some unlucky few will get fewer search results than they should. But we can optimize this in the future.

Searching

The way you connect to Elasticsearch is simply by a URL it looks something like this:

It’s an Elasticsearch cluster managed by Elastic running inside AWS. Our job is to make sure that we put the exact same URL in our GitHub Action (“the writer”) as we put it into our Django server (“the reader”).
In fact, we have 3 Elastic clusters: Prod, Stage, Dev.
And we have 2 Django servers: Prod, Stage.
So we just need to carefully make sure the secrets are set correctly to match the right environment.

Now, in the Django server, we just need to convert a request like GET /api/v1/search?q=foo&locale=fr (for example) to a query to send to Elasticsearch. We have a simple Django view function that validates the query string parameters, does some rate-limiting, creates a query (using elasticsearch-dsl) and packages the Elasticsearch results back to JSON.

How we make that query is important. In here lies the most important feature of the search; how it sorts results.

In one simple explanation, the sort order is a combination of popularity and “matchness”. The assumption is that most people want the popular content. I.e. they search for foreach and mean to go to /en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach not /en-US/docs/Web/API/NodeList/forEach both of which contains forEach in the title. The “popularity” is based on Google Analytics pageviews which we download periodically, normalize into a floating-point number between 1 and 0. At the time of writing the scoring function does something like this:

This seems to produce pretty reasonable results.

But there’s more to the “matchness” too. Elasticsearch has its own API for defining boosting and the way we apply is:

match phrase in the title: Boost = 10.0
match phrase in the body: Boost = 5.0
match in title: Boost = 2.0
match in body: Boost = 1.0

This is then applied on top of whatever else Elasticsearch does such as “Term Frequency” and “Inverse Document Frequency” (tf and if). This article is a helpful introduction.

We’re most likely not done with this. There’s probably a lot more we can do to tune this myriad of knobs and sliders to get the best possible ranking of documents that match.

Web UI

The last piece of the puzzle is how we display all of this to the user. The way it works is that developer.mozilla.org/$locale/search returns a static page that is blank. As soon as the page has loaded, it lazy-loads JavaScript that can actually issue the XHR request to get and display search results. The code looks something like this:

Learn Web Development Mdn

A lot of interesting details are omitted from this code snippet. You have to check it out for yourself to get a more up-to-date insight into how it actually works. But basically, the window.location (and pushState) query string drives the fetch() call and then all the component has to do is display the search results with some highlighting.

The /api/v1/search endpoint also runs a suggestion query as part of the main search query. This extracts out interest alternative search queries. These are filtered and scored and we issue “sub-queries” just to get a count for each. Now we can do one of those “Did you mean…”. For example: search for intersections.

In conclusion

There are a lot of interesting, important, and careful details that are glossed over here in this blog post. It’s a constantly evolving system and we’re constantly trying to improve and perfect the system in a way that it fits what users expect.

A lot of people reach MDN via a Google search (e.g. mdn array foreach) but despite that, nearly 5% of all traffic on MDN is the site-search functionality. The /$locale/search?... endpoint is the most frequently viewed page of all of MDN. And having a good search engine that’s reliable is nevertheless important. By owning and controlling the whole pipeline allows us to do specific things that are unique to MDN that other websites don’t need. For example, we index a lot of raw HTML (e.g. <video>) and we have code snippets that needs to be searchable.

Mdm Web Docs

Hopefully, the MDN site-search will elevate from being known to be very limited to something now that can genuinely help people get to the exact page better than Google can. Yes, it’s worth aiming high!

Mdn Learn Web Development

(Originally posted on personal blog)

Mozilla Web Dev

About Peter Bengtsson