12 個の投稿 / 0 new
#1 2017-08-12 17:41

Editing effects

Effects are easy to edit. You just have to start playing. You're not going to break anything. The absolute worst thing that can happen is that you need to reboot your computer or re-download an effect.

Effect files are just text files. You can open them in any text editor, including good old Notepad. I use Notepad++ because it has line numbers.

You'll see a few different names in folders: .FX, .FXSUB, .FXH. None of these much matter, they all work the same. .FX files tend to be primary files intended for loading directly in MME. .FXSUB files tend to effects loaded for render targets (those extra tabs in MME effect application dialog). FXH files tend to be like header files, including information used by a lot of different files.

Whenever you load a model into MMD, MME checks for any .FX files with the same name as the model, and if they exist, it loads them onto the model. (It also checks from an explicitly savable file format whose extension I forget.) So when you load ES.x into a scene, MME is smart enough to load ES.fx onto it. ES.fx then specifies some additional effects to load onto models in its own render targets.

Before you start editing, load your effect into a scene and make sure that MME is set to auto-reload. Now, every time you save an effect file being *directly* used by MME, it will reload the effect. You want this so you can see your changes. You also want this to be able to hear about your typos and bugs and whatnot.

Many effects have information intended to be be user-editable up near the top of their files. You're looking for "magic numbers." The standard here is to use #define statements near the top of the file to set magic numbers that could be other than they are. Look for something like,

#define PMD_SHADOWPOWER 1.1f

#define is a macro, or pre-processor directive. In English, what happens is a simple search and replace at load time. Every time (after the #define) that the effect sees "PMD_SHADOWPOWER" it instead plugs in "1.1f". This is important to understand because if you get an error, it may show up in the wrong place.

Say line 10 reads,

#define HUESHIFT 0.5f;

and line 216 reads,

HSV.x += HUESHIFT * scale;

What's going to happen is that line 216 turns into "HSV.x += 0.5f; * scale;" and the wrong syntax makes the compiler bug out. The error is on line 10, but the compiler sees the error as existing at line 216. (There shouldn't be the semi-colon on the #define.)

Okay, so to start out with, you just want to edit numbers (or perhaps filename strings) near the top of the files. Just ignore the language if it doesn't make sense. (This is HLSL, which is similar to C, and if you've programmed before, it should be easy to understand. If you haven't, well, everybody who programs learned how to do so somehow, and most of them by doing exactly this-- opening a file and messing with it.)

Just remember that programming is very specific. The compiler is smart, but it doesn't know what you mean to say. Capitalization matters. Proper punctuation matters. ")" is not the same as "}". ";" is not the same as ":". That's why you keep the effect open and autoloading, because you will make mistakes like this. I make them all the time. I make a hundred mistakes for every effect I write.

Still, to edit magic numbers, you're going to need to understand a few things.

"1.0f" is not the same thing as "1". Well, sometimes it is. Sometimes it isn't.

What's "3.6/2"? It's 2 of course! "2" is an integer. When the compiler reads that expression, it figures you're putting an integer in, so you want an integer out, and it rounds to the nearest one.

Most of the time you don't want an integer. You want a rational number, with decimal places. In programming these are called floating point values. "2.0f" is a way of specifying that, yes, you want the number to be 2, but you want it to be read as a floating point value. And so "3.6/2.0f" = 1.8, like you'd expect.

Effects use arrays of numbers all the time. For example, colors are a collection of 3 or 4 values: red, green, blue, and maybe alpha. So you're going to see lines like, "#define BLACK float4(0,0,0,1)". That's saying that BLACK is a collection of 4 floating point numbers. Red, green, and blue are all 0, and alpha is 1. Or "#define GREEN float3(0,1,0)" These are then accessed later in the program with lines like "Color.g = GREEN.g" or "Color.g = GREEN.y" (equivalent statements). You can read the elements of these with .xyzw or with .rgba notation, HLSL doesn't care, and there are times when you want a vector to be read as a color or a color as a vector.

You also may want to change image names (although it may be easier to rename your images to fit the effect.) Files are referenced by strings. String are surrounded by quotes. Directory structure assumes the main effect file as the starting point. So if ES.fx is in effects/ES, and noise.png is in effects/ES/tex, then if you want ES.fx to read noise, you'd probably use a line like

#define NOISETEX "tex/noise.png"

Forward slash, not back slash.

I can go into tons more detail if anybody is more interested. I write my own effects from scratch now. I'm not a HLSL expert yet but I can do a lot. Won't answer questions about specific effects, though, it usually means that I dig through uninteresting code for a long time to figure out the answer.

2017-08-12 19:45

Thank You bandages very good tutorial easy to understand i like to put my models stages and effects in to a bluray data disk then copy to my desk top when editing well thats just me tho 40 cents for a bd is cheap. l like easy to understand tutorials like yours and pr0nb0tz also bobbeebs oops cant forget jajandulu. well to many helpful people to name but their help is always appreciated.

2017-08-12 23:52

This was interesting. Thank you.

2017-08-13 07:46
What's "3.6/2"? It's 2 of course! "2" is an integer. When the compiler reads that expression, it figures you're putting an integer in, so you want an integer out, and it rounds to the nearest one.

Are you sure about that one? I've used integers with floating point calculations and have never had any problems so long as the other number is a float. Pretty much every compiler and language I've ever worked with follows the rule to cast the integer to a float first if it's being added/multiplied/divided/etc with a float.

I think what you're thinking of is when you're doing division and both numbers are integers like dividing 3 by 2 and then the operation isn't exactly division but the modular div command. In that case it would give 1 when you would normally expect 1.5 because only 1 instance of 2 fits into 3 without going over. Also I'm pretty sure if 3.6/2 would be rounded because of integer casting it would be rounded down to 1 not up to 2 because it's not actually rounded but truncated meaning the decimal part is just chopped off regardless of how close it is to the next number.

I know that's how it works in most languages but maybe it doesn't work like that in HLSL so if you could show me how you determined that I could try it for myself to confirm or deny it.

2017-08-13 07:55

> Are you sure about that one? I've used integers with floating point calculations and have never had any problems so long as the other number is a float. Pretty much every compiler and language I've ever worked with follows the rule to cast the integer to a float first if it's being added/multiplied/divided/etc with a float.

No I'm not sure, that example's untested, just letting beginners know to be careful. It may very well be that I was dividing an integer by an integer when I realized that this was an issue. Since then, I make everything a float, and so I'm unfamiliar with the exact behavior of integer math in HLSL. I don't think there's any reason to use integers in HLSL, not even for counts. I've read that all integer math is done with floating point calculations anyways-- all integers can do is slow it down. (Not by enough to matter of course.)

2017-08-14 09:12

i'd like to edit raycast .fxsub files however theyre are enormous amount of variables which could be changed. without knowing what does what, its going to take days to figure out.

any clues to where to find out what each thing does, boss? just an example out of the many http://puu.sh/x9D5T/90db35e1a5.png

2017-08-14 16:49

> i'd like to edit raycast .fxsub files however theyre are enormous amount of variables which could be changed. without knowing what does what, its going to take days to figure out.

"Days to figure out" doesn't strike me as a long time. You shouldn't expect to get good results from anything in minutes or hours. And starting out by editing the hugest effect in MMD is probably not the best bet. If you want to learn, learn on something simpler. If you just want to get stuff done, pick up some extra hours at McDonalds and hire somebody else to do it, that'll be faster.

As you do more, you'll acquire a larger vocabulary that will help you understand possible outcomes based on the names of variables. Before you know what specular is, something named SPECULARFACTOR is a mystery; afterwards, you have a very good idea of what it's going to do without looking at any code.

On the screenshot you gave, you could do a search (through all the .fx style files used by Raycast, or, easier, on its Github page) for #define HDR_STAR_MODE to see where that gets set, and by what inputs; you could search for ColorCoeff1st etc and see where those are used, and how. I encourage you to do that. That's exactly how you get started actually learning to code effects yourself.

2017-08-15 00:25

I was reading some .fx files and I saw code that looked something like this:

float Extent
   string UIName = "Extent";
   string UIWidget = "Slider";
   bool UIVisible =  true;
   float UIMin = 0.00;
   float UIMax = 0.01;
> = float( 0.007 );

The purpose of the variable isn't important here but what I'm wondering is why I don't see any of these UI Widgets while editing in MMD. It's clear the stuff in the angle brackets is trying to define a GUI window to let the user control its value but I'm not seeing these sliders anywhere. Is this just a deprecated feature of MMD or are there other programs besides MMD that use .fx files or something?

2017-08-15 01:59

>The purpose of the variable isn't important here but what I'm wondering is why I don't see any of these UI Widgets while editing in MMD.

Because MMD doesn't support them. My understanding is that these widgets were created for use with MMM-only features (although I'm not familiar with MMM).

Note that this is a great place to get started editing files. You can expose all of these variables to MMD. For instance, try deleting (or commenting out) that section defining Extent and replace it with the following,

static float Extent = 0.007f;

Now, depending on how Extent is used by the effect, that should give you the same value. Now, you can edit that text file and change Extent to be whatever you want. If MMD is open, you should be able to see the changes as you make them.

The step after that is to create a controller-- that is, to attach Extent to a bone, morph, or accessory property, so that you can slide it around in MMD and animate it-- even OP it to another bone to automate animation. This is well within reach of even beginners, a great starting project.

2017-08-15 03:31

Akomni, you might want to take a look at the ray.conf file first. That one is much easier to edit. That can affect things like reflections and lens flare.

2017-08-15 08:46
bandages wrote:

yeah the learning curve is huge. and yea, a few days really was an understatement. don't really know what i was expecting.. a magical shortcut maybe. alas not everything is so easy. thanks for the response though. i'll have to set aside a lot of time to figure raycast out though. as you mentioned in a previous post, if the effect updates in real time when using it in mmd, that would make things a hell of a lot faster.

Pr0nb0tz wrote:

unfortunately the .conf file options are very limited besides the basic "turn x on/off". i kind of want to delve into making it a bit more customized to my tastes, as i imagine the things you could do with ray are vast.

2017-08-15 09:30

Can't sleep, so a bit more info.

Comments are important to understand. Comments are a way to include plain English information about code. Which is useful not just for other people, but for your future self, a few months down the road, trying to figure what the hell you were thinking. Comments are also a way to nullify code without deleting it.

//this line is a comment because of the two forward slashes at the beginning of the line

/*this series
of lines is a comment because of the way
it's delineated

/*these nested comments
do this*/
will give you an error*/

If you run into an error that's not just a typo someplace, you comment out code until it works again. Then you uncomment until it stops working. That's how a ton of bugs are solved.

#include is another pre-processor directive. Like #define, it's a dumb word processing action. It looks for the file listed after the #include and pastes it in place of the #include statement. So you'll see lines like

#include "library/matrix.fxh"

which looks for that file and pastes what it contains right there.

An effect is ultimately composed of one or more techniques containing one or more passes. Techniques are programs and passes are subprograms. MMD uses several techniques in its default shader. If you don't specify what a particular technique is composed of, and it is a technique used by MMD by default, MME will assume you want to use MMD's default technique.

So for example, ZplotTec is MMD's default technique for writing the distance from the light to the nearest shadow caster, from the perspective of the light. This is used for shadowing. If you're making an effect and you don't specify a ZplotTec (nobody ever does), that object will cast shadows normally. But if you add to your file,

technique ZplotTec {

then that technique becomes an empty technique-- you overwrite the default. And your objects stop casting shadows.

There are several default techniques. These are EdgeTec (draws edge line), ShadowTec (draws ground shadow), ZPlotTec (draws to shadow buffer), MainTec0-7 (techniques for drawing various kinds of objects in no-shadow mode), MainTecBS0-7 (techniques for drawing various kinds of objects in shadowed mode). You can recreate any of these you want. Not all model/material values are available in all techniques. You can only get edge color in EdgeTec, for example.

If you don't use one of these names, it doesn't matter what your technique is named. Use anything you want.

Passes are typically composed of a vertex shader and a pixel shader. A vertex shader is run for every vertex. From the output of this, the pixel shader is basically run for every pixel inside of the triangle composed by three vertices. The vertex shader sends its output to the pixel shader, and for each pixel, the output is interpolated between the three vertices. The most important output is Pos ( ":POSITION") which is the screen position of the vertex, interpolated to become the screen position of each of that vertex's pixels. Pos is magic, and has rules that are unlike rules for other variables, so be careful if you want to play with it.

Pixel shaders take that input and output : COLOR. There's no magic here.

Passes can be flagged in various ways-- to avoid depth culling, to change from alpha blend to add mode, etc.

When a shader writes to a pixel, it keeps a running depth buffer that you can't access, if you want one make your own. It checks depth first. If the depth (Pos.z) of the current pixel is higher, it doesn't write anything (assuming it's depth buffered). After it writes a color, it writes a new depth. Regardless of alpha. That's why draw order matters.

In most effects, all that other stuff in the file is just setup for the vertex shader and the pixel shader-- defining values from those sent by MMD, telling MMD about textures and how to sample them, doing some basic arithmetic that will used identically by all shaders in the file, and creating functions. There are a few exceptions.

In some files you will see a section that starts with

float Script : STANDARDSGLOBAL <

This creates some important information. The most important bit is whether this is a post-process (writes to the screen after other models, like DOF), a pre-process (will get overwritten, never seen it used by anybody else but you can use it to setup procedural textures, could probably use it to make a fancy skydome) or standard/object (the default, that's not the technical name, I just don't remember off the top of my head; it sets up a way of rendering a specific object, and is used by effects like Working Floor.)

Sometimes effects set up RENDERTARGET textures. These are those extra tabs you find in the MME effect dialogue. For example, Working Floor sets one of these up to re-render every object from a different perspective, creating a texture that it can use to draw the actual Working Floor accessory. Sometimes, these can be shared, so that multiple effects can use them. As with Excellent Shadow, which shares its shadowmap for the main render window. Typically these specify effects to load. It does this via filename and wildcards usually. So Excellent shadow can hide "*.x" from its shadowmap if it wants and show "*.pmx" if it wants. And you can have an effect involving the interaction of multiple models/effects, in which case this can be handy.

You may find that an effect's technique is composed of a huge script. For instance, AutoLuminous writes a bunch stuff to render targets, then it blurs them, then it blurs them some more differently, then blah blah blah (AL is total overkill). If you want to do a bunch of stuff, just open up AL and start copying stuff and renaming the passes so you can write your own. Basically, for each section of the script, it chooses a render target, it sets up depth and clears color/depth, and then it specifies a pass to use on that render target.

There are a few things to keep in mind. MME uses DirectX 9. It is limited to shader model 3.0. It cannot read textures in the vertex shader. It cannot read mipmapped textures inside of loops. It cannot do string operations to choose between pre-generated texture files. There are a limited number of constants you can stick in an FX file. (If you don't need them in the VS, stick them in a texture instead.) There are a limited number of textures that can be accessed by any particular pass. There are functions that are available in later shader models that are unavailable in MME.

Shaders can do a lot. They cannot write to two images at the same time. They cannot read an image at the same time that they write to it. If they want to remember anything from frame to frame, they have to draw onto a picture and then read that picture the next frame. Independent pixel shaders cannot talk to each other (well, they can poll their immediate neighbors acting on the exact same triangle, but that's it.) It can be tricky thinking about how to do what you want to do inside a massively parallel GPU. But it's usually possible. Somehow.

Don't optimize. The compiler is smarter than you are. Ignore the ten year old optimization advice you might read, it's way out of date. Use a loop if you want, use an if statement if it will make your code more readable. Pow() is the least of your concerns. Don't go creating crazy unreadable ternary operator expressions. The compiler will make your code as fast as it can be without being coded in assembly. That's why it takes so long to load, it's doing a shitload of thinking about how to compile the effect.