diff options
author | Luke Shumaker <LukeShu@sbcglobal.net> | 2014-01-28 09:50:25 -0500 |
---|---|---|
committer | Luke Shumaker <LukeShu@sbcglobal.net> | 2014-01-28 09:50:25 -0500 |
commit | 5744df39e15f85c6cc8a9faf8924d77e76d2b216 (patch) | |
tree | a8c8dd40a94d1fa0d5377566aa5548ae55a163da /includes/filebackend/README | |
parent | 4bb2aeca1d198391ca856aa16c40b8559c68daec (diff) | |
parent | 224b22a051051f6c2e494c3a2fb4adb42898e2d1 (diff) |
Merge branch 'archwiki'
Conflicts:
extensions/FluxBBAuthPlugin.php
extensions/SyntaxHighlight_GeSHi/README
extensions/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.class.php
extensions/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.i18n.php
extensions/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.php
extensions/SyntaxHighlight_GeSHi/geshi/docs/CHANGES
extensions/SyntaxHighlight_GeSHi/geshi/docs/THANKS
extensions/SyntaxHighlight_GeSHi/geshi/docs/TODO
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/AbstractClass.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/AbstractClass_logo.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/AbstractMethod.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/AbstractPrivateClass.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/AbstractPrivateClass_logo.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/AbstractPrivateMethod.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Class.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Class_logo.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Constant.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Constructor.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Destructor.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Function.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Global.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/I.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Index.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Interface.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Interface_logo.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/L.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Lminus.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Lplus.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Method.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Page.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Page_logo.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/PrivateClass.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/PrivateClass_logo.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/PrivateMethod.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/PrivateVariable.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/StaticMethod.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/StaticVariable.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/T.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Tminus.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Tplus.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/Variable.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/blank.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/class_folder.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/file.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/folder.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/function_folder.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/next_button.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/next_button_disabled.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/package.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/package_folder.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/previous_button.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/previous_button_disabled.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/private_class_logo.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/tutorial.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/tutorial_folder.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/api/media/images/up_button.png
extensions/SyntaxHighlight_GeSHi/geshi/docs/geshi-doc.html
extensions/SyntaxHighlight_GeSHi/geshi/docs/geshi-doc.txt
extensions/SyntaxHighlight_GeSHi/geshi/geshi.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/4cs.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/6502acme.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/6502kickass.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/6502tasm.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/68000devpac.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/abap.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/actionscript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/actionscript3.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/ada.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/algol68.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/apache.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/applescript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/apt_sources.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/asm.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/asp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/autoconf.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/autohotkey.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/autoit.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/avisynth.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/awk.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/bascomavr.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/bash.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/basic4gl.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/bf.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/bibtex.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/blitzbasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/bnf.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/boo.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/c.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/c_loadrunner.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/c_mac.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/caddcl.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cadlisp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cfdg.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cfm.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/chaiscript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cil.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/clojure.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cmake.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cobol.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/coffeescript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cpp-qt.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cpp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/csharp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/css.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/cuesheet.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/d.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/dcs.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/delphi.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/diff.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/div.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/dos.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/dot.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/e.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/ecmascript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/eiffel.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/email.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/epc.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/erlang.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/euphoria.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/f1.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/falcon.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/fo.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/fortran.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/freebasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/fsharp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/gambas.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/gdb.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/genero.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/genie.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/gettext.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/glsl.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/gml.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/gnuplot.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/go.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/groovy.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/gwbasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/haskell.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/hicest.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/hq9plus.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/html4strict.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/html5.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/icon.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/idl.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/ini.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/inno.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/intercal.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/io.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/j.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/java.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/java5.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/javascript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/jquery.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/kixtart.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/klonec.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/klonecpp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/latex.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/lb.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/lisp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/llvm.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/locobasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/logtalk.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/lolcode.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/lotusformulas.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/lotusscript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/lscript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/lsl2.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/lua.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/m68k.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/magiksf.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/make.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/mapbasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/matlab.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/mirc.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/mmix.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/modula2.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/modula3.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/mpasm.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/mxml.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/mysql.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/newlisp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/nsis.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/oberon2.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/objc.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/objeck.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/ocaml-brief.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/ocaml.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/oobas.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/oracle11.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/oracle8.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/oxygene.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/oz.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/pascal.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/pcre.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/per.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/perl.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/perl6.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/pf.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/php-brief.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/php.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/pic16.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/pike.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/pixelbender.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/pli.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/plsql.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/postgresql.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/povray.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/powerbuilder.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/powershell.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/proftpd.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/progress.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/prolog.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/properties.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/providex.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/purebasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/pycon.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/python.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/q.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/qbasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/rails.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/rebol.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/reg.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/robots.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/rpmspec.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/rsplus.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/ruby.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/sas.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/scala.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/scheme.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/scilab.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/sdlbasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/smalltalk.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/smarty.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/sql.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/systemverilog.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/tcl.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/teraterm.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/text.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/thinbasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/tsql.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/typoscript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/unicon.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/uscript.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/vala.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/vb.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/vbnet.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/verilog.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/vhdl.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/vim.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/visualfoxpro.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/visualprolog.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/whitespace.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/whois.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/winbatch.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/xbasic.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/xml.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/xorg_conf.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/xpp.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/yaml.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/z80.php
extensions/SyntaxHighlight_GeSHi/geshi/geshi/zxbasic.php
Diffstat (limited to 'includes/filebackend/README')
-rw-r--r-- | includes/filebackend/README | 208 |
1 files changed, 208 insertions, 0 deletions
diff --git a/includes/filebackend/README b/includes/filebackend/README new file mode 100644 index 00000000..569f3376 --- /dev/null +++ b/includes/filebackend/README @@ -0,0 +1,208 @@ +/*! +\ingroup FileBackend +\page file_backend_design File backend design + +Some notes on the FileBackend architecture. + +\section intro Introduction + +To abstract away the differences among different types of storage media, +MediaWiki is providing an interface known as FileBackend. Any MediaWiki +interaction with stored files should thus use a FileBackend object. + +Different types of backing storage media are supported (ranging from local +file system to distributed object stores). The types include: + +* FSFileBackend (used for mounted file systems) +* SwiftFileBackend (used for Swift or Ceph Rados+RGW object stores) +* FileBackendMultiWrite (useful for transitioning from one backend to another) + +Configuration documentation for each type of backend is to be found in their +__construct() inline documentation. + + +\section setup Setup + +File backends are registered in LocalSettings.php via the global variable +$wgFileBackends. To access one of those defined backends, one would use +FileBackendStore::get( <name> ) which will bring back a FileBackend object +handle. Such handles are reused for any subsequent get() call (via singleton). +The FileBackends objects are caching request calls such as file stats, +SHA1 requests or TCP connection handles. + +\par Note: +Some backends may require additional PHP extensions to be enabled or can rely on a +MediaWiki extension. This is often the case when a FileBackend subclass makes use of an +upstream client API for communicating with the backing store. + + +\section fileoperations File operations + +The MediaWiki FileBackend API supports various operations on either files or +directories. See FileBackend.php for full documentation for each function. + + +\subsection reading Reading + +The following basic operations are supported for reading from a backend: + +On files: +* stat a file for basic information (timestamp, size) +* read a file into a string or several files into a map of path names to strings +* download a file or set of files to a temporary file (on a mounted file system) +* get the SHA1 hash of a file +* get various properties of a file (stat information, content time, mime information, ...) + +On directories: +* get a list of files directly under a directory +* get a recursive list of files under a directory +* get a list of directories directly under a directory +* get a recursive list of directories under a directory + +\par Note: +Backend handles should return directory listings as iterators, all though in some cases +they may just be simple arrays (which can still be iterated over). Iterators allow for +callers to traverse a large number of file listings without consuming excessive RAM in +the process. Either the memory consumed is flatly bounded (if the iterator does paging) +or it is proportional to the depth of the portion of the directory tree being traversed +(if the iterator works via recursion). + + +\subsection writing Writing + +The following basic operations are supported for writing or changing in the backend: + +On files: +* store (copying a mounted file system file into storage) +* create (creating a file within storage from a string) +* copy (within storage) +* move (within storage) +* delete (within storage) +* lock/unlock (lock or unlock a file in storage) + +The following operations are supported for writing directories in the backend: +* prepare (create parent container and directories for a path) +* secure (try to lock-down access to a container) +* publish (try to reverse the effects of secure) +* clean (remove empty containers or directories) + + +\subsection invokingoperation Invoking an operation + +Generally, callers should use doOperations() or doQuickOperations() when doing +batches of changes, rather than making a suite of single operation calls. This +makes the system tolerate high latency much better by pipelining operations +when possible. + +doOperations() should be used for working on important original data, i.e. when +consistency is important. The former will only pipeline operations that do not +depend on each other. It is best if the operations that do not depend on each +other occur in consecutive groups. This function can also log file changes to +a journal (see FileJournal), which can be used to sync two backend instances. +One might use this function for user uploads of file for example. + +doQuickOperations() is more geared toward ephemeral items that can be easily +regenerated from original data. It will always pipeline without checking for +dependencies within the operation batch. One might use this function for +creating and purging generated thumbnails of original files for example. + + +\section consistency Consistency + +Not all backing stores are sequentially consistent by default. Various FileBackend +functions offer a "latest" option that can be passed in to assure (or try to assure) +that the latest version of the file is read. Some backing stores are consistent by +default, but callers should always assume that without this option, stale data may +be read. This is actually true for stores that have eventual consistency. + +Note that file listing functions have no "latest" flag, and thus some systems may +return stale data. Thus callers should avoid assuming that listings contain changes +made my the current client or any other client from a very short time ago. For example, +creating a file under a directory and then immediately doing a file listing operation +on that directory may result in a listing that does not include that file. + + +\section locking Locking + +Locking is effective if and only if a proper lock manager is registered and is +actually being used by the backend. Lock managers can be registered in LocalSettings.php +using the $wgLockManagers global configuration variable. + +For object stores, locking is not generally useful for avoiding partially +written or read objects, since most stores use Multi Version Concurrency +Control (MVCC) to avoid this. However, locking can be important when: +* One or more operations must be done without objects changing in the meantime. +* It can also be useful when a file read is used to determine a file write or DB change. + For example, doOperations() first checks that there will be no "file already exists" + or "file does not exist" type errors before attempting an operation batch. This works + by stating the files first, and is only safe if the files are locked in the meantime. + +When locking, callers should use the latest available file data for reads. +Also, one should always lock the file *before* reading it, not after. If stale data is +used to determine a write, there will be some data corruption, even when reads of the +original file finally start returning the updated data without needing the "latest" +option (eventual consistency). The "scoped" lock functions are preferable since +there is not the problem of forgetting to unlock due to early returns or exceptions. + +Since acquiring locks can fail, and lock managers can be non-blocking, callers should: +* Acquire all required locks up font +* Be prepared for the case where locks fail to be acquired +* Possible retry acquiring certain locks + +MVCC is also a useful pattern to use on top of the backend interface, because operations +are not atomic, even with doOperations(), so doing complex batch file changes or changing +files and updating a database row can result in partially written "transactions". Thus one +should avoid changing files once they have been stored, except perhaps with ephemeral data +that are tolerant of some degree of inconsistency. + +Callers can use their own locking (e.g. SELECT FOR UPDATE) if it is more convenient, but +note that all callers that change any of the files should then go through functions that +acquire these locks. For example, if a caller just directly uses the file backend store() +function, it will ignore any custom "FOR UPDATE" locks, which can cause problems. + +\section objectstore Object stores + +Support for object stores (like Amazon S3/Swift) drive much of the API and design +decisions of FileBackend, but using any POSIX compliant file systems works fine. +The system essentially stores "files" in "containers". For a mounted file system +as a backing store, "files" will just be files under directories. For an object store +as a backing store, the "files" will be objects stored in actual containers. + + +\section file_obj_diffs File system and Object store differences + +An advantage of object stores is the reduced Round-Trip Times. This is +achieved by avoiding the need to create each parent directory before placing a +file somewhere. It gets worse the deeper the directory hierarchy is. Another +advantage of object stores is that object listings tend to use databases, which +scale better than the linked list directories that file sytems sometimes use. +File systems like btrfs and xfs use tree structures, which scale better. +For both object stores and file systems, using "/" in filenames will allow for the +intuitive use of directory functions. For example, creating a file in Swift +called "container/a/b/file1" will mean that: +- a "directory listing" of "container/a" will contain "b", +- and a "file listing" of "b" will contain "file1" + +This means that switching from an object store to a file system and vise versa +using the FileBackend interface will generally be harmless. However, one must be +aware of some important differences: + +* In a file system, you cannot have a file and a directory within the same path + whereas it is possible in an object stores. Calling code should avoid any layouts + which allow files and directories at the same path. +* Some file systems have file name length restrictions or overall path length + restrictions that others do not. The same goes with object stores which might + have a maximum object length or a limitation regarding the number of files + under a container or volume. +* Latency varies among systems, certain access patterns may not be tolerable for + certain backends but may hold up for others. Some backend subclasses use + MediaWiki's object caching for serving stat requests, which can greatly + reduce latency. Making sure that the backend has pipelining (see the + "parallelize" and "concurrency" settings) enabled can also mask latency in + batch operation scenarios. +* File systems may implement directories as linked-lists or other structures + with poor scalability, so calling code should use layouts that shard the data. + Instead of storing files like "container/file.txt", one can store files like + "container/<x>/<y>/file.txt". It is best if "sharding" optional or configurable. + +*/ |